-- Logs begin at Mon 2022-06-06 19:40:52 UTC, end at Mon 2022-06-06 21:01:00 UTC. -- Jun 06 20:22:43.590386 dv-cinder-844879-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 06 20:22:43.597610 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 6 20:22:43 2022] *** Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: nodename: dv-cinder-844879-1-iscsi Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: machine: x86_64 Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: clock source: unix Jun 06 20:22:43.599601 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: pcre jit disabled Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: detected number of CPU cores: 4 Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: current working directory: / Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: detected binary path: /usr/bin/uwsgi-core Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: your processes number limit is 47680 Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: your memory page size is 4096 bytes Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: detected max file descriptor number: 2048 Jun 06 20:22:43.600225 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: lock engine: pthread robust mutexes Jun 06 20:22:43.600838 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: thunder lock: enabled Jun 06 20:22:43.601032 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 06 20:22:43.601119 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 06 20:22:43.645485 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: Python main interpreter initialized at 0x55d748e969c0 Jun 06 20:22:43.645485 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: python threads support enabled Jun 06 20:22:43.645485 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: your server socket listen backlog is limited to 100 connections Jun 06 20:22:43.645485 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: your mercy for graceful operations on workers is 90 seconds Jun 06 20:22:43.645836 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: mapped 403077 bytes (393 KB) for 2 cores Jun 06 20:22:43.645920 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: *** Operational MODE: preforking *** Jun 06 20:22:43.646187 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: *** uWSGI is running in multiple interpreter mode *** Jun 06 20:22:43.646187 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: spawned uWSGI master process (pid: 80381) Jun 06 20:22:43.646389 dv-cinder-844879-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 06 20:22:43.647463 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: spawned uWSGI worker 1 (pid: 80392, cores: 1) Jun 06 20:22:43.647463 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: spawned uWSGI worker 2 (pid: 80393, cores: 1) Jun 06 20:22:43.647463 dv-cinder-844879-1-iscsi devstack@keystone.service[80381]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 06 20:22:49.329435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 20:22:49.329435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: from cryptography.utils import int_from_bytes Jun 06 20:22:49.330184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 20:22:49.330184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: from cryptography.utils import int_from_bytes Jun 06 20:22:49.330606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 20:22:49.330606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: from cryptography.utils import int_from_bytes Jun 06 20:22:49.330734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 06 20:22:49.330734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: from cryptography.utils import int_from_bytes Jun 06 20:22:49.379375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 20:22:49.379528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 06 20:22:49.379603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 20:22:49.379751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 06 20:22:49.379834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] command line args: [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 20:22:49.379904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] command line args: [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 06 20:22:49.380033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 20:22:49.380140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 06 20:22:49.380218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ================================================================================ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 20:22:49.380310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ================================================================================ {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 06 20:22:49.380455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] admin_token = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.380548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] admin_token = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.380702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_dir = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.380801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] config_dir = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.380878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.380954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.381139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] config_source = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.381304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] config_source = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.381392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] debug = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.381490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] debug = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.381568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.382788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.382888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.382956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_config_append = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_config_append = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.383940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_dir = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_dir = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_file = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.384961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.385912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386314 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.386968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.41/identity {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] publish_errors = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] publish_errors = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.387933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] standard_threads = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] standard_threads = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.388980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] use_journal = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_journal = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] use_json = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] use_stderr = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.389971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_json = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.390089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] use_syslog = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.390207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_stderr = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.390207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] use_syslog = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.390425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.390535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 06 20:22:49.390645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.390761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.390872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.390991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.391940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.392962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.393943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.394988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.395931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.396959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.397993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398865 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.398965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.399917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.400938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.401971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402545 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.402894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.403967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.404906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.405906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.406943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.407979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.408926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.409945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.410068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.410181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.410301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.410414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.410534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.410897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.411975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.412900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.413909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.414936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.external = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.external = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.415909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.password = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.password = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] auth.token = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] auth.token = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.416948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.417991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.418887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.419942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.420970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.421951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.422972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.423736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.423854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.423968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.424911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.425958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.426074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.426074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.427155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.427416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.427532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.427709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.427832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.427995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.428960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.429966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.430935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.431888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.432930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433654 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.433994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.434915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.435899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.436922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.437956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.438069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.438181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.438294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.438408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.438529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.438705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.439337 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.439451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.439564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.439696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.439827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.439943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.440977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.441903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.442931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.443936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.444987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445451 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.445901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.446948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.447919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.448947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] role.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.449967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] role.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.450999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.451932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.452967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.453888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.454953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.455987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.456774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.457944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.458975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.459918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.460950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461174 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.461988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.connection = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.462888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.463989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.464908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.465959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80392) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 20:22:49.466648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 06 20:22:49.466858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80393) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 06 20:22:49.495301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.495425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.495513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.495589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.495716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.495801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.504513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.504615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.531988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.532222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.532304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.533007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.585838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.586051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.634714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.634945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.635275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.635511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.701997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.702132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.702214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.702348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.702430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.726861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.727003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.727095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.727218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.727301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.727433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.727515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.738857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.739006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.751578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.751806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.752028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.752245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.752458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.752669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.752880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80392) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.753507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.753742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.753969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.754193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.754410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.754622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.754836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80393) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 06 20:22:49.770086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.770291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 20:22:49.770374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.770579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 20:22:49.770660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.770804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 20:22:49.770913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 06 20:22:49.771328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.771945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.772118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.783274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.783517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.783599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.783814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.783897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.784083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.784178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.784363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.792743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.792894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.796001 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.796187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.796269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.796467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.796548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.796748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.796836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.797964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.798633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.798711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.798875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.798957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.799137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.799217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.799399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.825320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.825662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.826907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.827183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.827264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.827525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.832555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.832830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.832938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.833104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.833214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.833492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.840341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.840621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.840746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.841026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.845726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.846008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.846090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.846283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.846363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.846557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.846723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.846827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.846908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.847094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.847174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.847355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.847436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.847617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.847762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.848017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.866750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.867020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.867102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.867373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.871655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.871989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.875375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.875683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.875766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.876008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.880775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.881030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.884389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.884631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.888008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.888278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.888374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.888568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.888649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.888847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.888925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.889120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.889211 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.889393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.889484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.889672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.909376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.909652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.909792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.909914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.910073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.910156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.910340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.910602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.918255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.918751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.921437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.921713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.921988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.922364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.924814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.925092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.925367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.925502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.925642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.925782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.925923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.926061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.926340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.926615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.939315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.939593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.939971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.940056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.940253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.940335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.940535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.940629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.940817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.940901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.941092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.941172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.941365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.941642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.959312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.959951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.962558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.963193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.965999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.966679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.967600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.968301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.969318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.969599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.969919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.970520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.970801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.971081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.977725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.978420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.979322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.980066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.982655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.982940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.983228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.983404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.983486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.983714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.984009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.984281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.992543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.992825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.993098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.993373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:49.993657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:49.993936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.001198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80392) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:50.001477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.001793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.001928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80393) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 06 20:22:50.002011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.002207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.002289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.002485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.002576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.002758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.002841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.003950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80392) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.004902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.005173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.005426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80393) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 06 20:22:50.043379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d748e969c0 pid: 80392 (default app) Jun 06 20:22:50.044206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55d748e969c0 pid: 80393 (default app) Jun 06 20:22:50.046726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757824-e5d5-47a2-92ea-5d5a9a0a7627 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:22:50.047015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757824-e5d5-47a2-92ea-5d5a9a0a7627 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:22:50.047289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757824-e5d5-47a2-92ea-5d5a9a0a7627 None None] PATH_INFO: `/v3` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:22:50.052045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1/1] 10.222.0.41 () {50 vars in 791 bytes} [Mon Jun 6 20:22:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:22:58.932939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:22:58.933504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:22:58.933504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:22:58.937769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1/2] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:22:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:22:58.943371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:22:58.943823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:22:58.943949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:22:59.010900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: INFO dbcounter [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] Registered counter for database keystone Jun 06 20:22:59.031789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG oslo_db.sqlalchemy.engines [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80392) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 20:22:59.038296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writer thread running {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 20:22:59.316961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG passlib.handlers.bcrypt [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80392) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 20:22:59.317391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG passlib.handlers.bcrypt [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80392) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 20:22:59.372525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:22:59.374103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:22:59.496160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 2/3] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:22:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 554 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 20:22:59.696577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:22:59.763807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: INFO dbcounter [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] Registered counter for database keystone Jun 06 20:22:59.783741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG oslo_db.sqlalchemy.engines [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80393) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 06 20:22:59.790093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writer thread running {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 06 20:23:00.010068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c313f1-8ee2-4f9e-8549-148724e3f341 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:00.014301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:00.014893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:00.015174 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:00.015444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:00.023908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:00.075639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING keystone.common.rbac_enforcer.enforcer [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 20:23:00.075998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c12c11-ca5f-44ab-bb37-996b47f51214 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:00.085793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 2/4] 10.222.0.41 () {58 vars in 1324 bytes} [Mon Jun 6 20:22:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 20:23:00.091766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:00.105913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37de34-ed74-4261-8438-3da927f3d6ec None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:00.110415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:00.111540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:00.111858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:00.112141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] PATH_INFO: `/v3/projects/284b2c60254e43d98184156216a03e15` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:00.113648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] RBAC: Authorizing `identity:get_project(project_id=284b2c60254e43d98184156216a03e15)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:00.164013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 06 20:23:00.164356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732e87b-7367-4848-b10f-208465362cef admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:00.170151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 3/5] 10.222.0.41 () {56 vars in 1207 bytes} [Mon Jun 6 20:23:00 2022] GET /identity/v3/projects/284b2c60254e43d98184156216a03e15 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:02.359184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d209de-e2aa-40fb-a0e2-19bd70452d0d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:02.359754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d209de-e2aa-40fb-a0e2-19bd70452d0d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:02.359754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d209de-e2aa-40fb-a0e2-19bd70452d0d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:02.363867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 3/6] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:23:02.369140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:02.369561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:02.369708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:02.388577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:02.390092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:02.398566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 4/7] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:02 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 06 20:23:02.420912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d209de-e2aa-40fb-a0e2-19bd70452d0d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:02.432867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d209de-e2aa-40fb-a0e2-19bd70452d0d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:02.436974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0771306-60ff-4da4-b4a5-eaf4d8ffbced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:02.437584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0771306-60ff-4da4-b4a5-eaf4d8ffbced admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:02.437867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0771306-60ff-4da4-b4a5-eaf4d8ffbced admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:02.438147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0771306-60ff-4da4-b4a5-eaf4d8ffbced admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:02.446128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0771306-60ff-4da4-b4a5-eaf4d8ffbced admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:02.447902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0771306-60ff-4da4-b4a5-eaf4d8ffbced admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:02.457005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 4/8] 10.222.0.41 () {58 vars in 1324 bytes} [Mon Jun 6 20:23:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:23:02.462149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:02.470995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f618e6-c3f4-4596-8354-c01687f86deb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:02.475348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81af45e-878d-4f5b-8164-63041a1ea98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:02.476230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81af45e-878d-4f5b-8164-63041a1ea98d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:02.476517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81af45e-878d-4f5b-8164-63041a1ea98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:02.476794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81af45e-878d-4f5b-8164-63041a1ea98d admin admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:02.478176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81af45e-878d-4f5b-8164-63041a1ea98d admin admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:02.480096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81af45e-878d-4f5b-8164-63041a1ea98d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:02.481584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 5/9] 10.222.0.41 () {56 vars in 1198 bytes} [Mon Jun 6 20:23:02 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:04.758659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7e7a-e519-480c-8708-233b45cdcb07 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:04.759202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7e7a-e519-480c-8708-233b45cdcb07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:04.759202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432e7e7a-e519-480c-8708-233b45cdcb07 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:04.760581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 5/10] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:04.766136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:04.766551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:04.766695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:04.767383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5550a-d41f-478b-81a9-0fe424081aaf None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:04.767697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5550a-d41f-478b-81a9-0fe424081aaf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:04.767988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5550a-d41f-478b-81a9-0fe424081aaf None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:04.769239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 6/11] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:23:04.774409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:04.774701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:04.774981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:04.783591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:04.784794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:04.814620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG passlib.handlers.bcrypt [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80393) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 06 20:23:04.815046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG passlib.handlers.bcrypt [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80393) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 06 20:23:04.867114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:04.867895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 6/12] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:04.868753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:04.890567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:04.915286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:04.968284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 7/13] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:04.990647 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:04.994003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9264a6-a413-41c7-9793-68ac9b724852 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.027360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.028125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.028410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.028690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.040515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:05.051700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:05.052692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:05.053539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 7/14] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:05.058859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:05.089661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01430cf-a275-4851-bd6e-e0b928299762 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.091894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:05.125023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.125902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.126188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.126463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.138977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:05.156889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e70426-8a48-488d-8492-a24f1da39b26 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.190036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.190690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.190974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.191250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.192014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:05.192141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.192318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.193662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.193748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.205582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 8/15] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:05.211063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:05.293035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 8/16] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 20:23:05.298461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:05.301195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:05.309877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c15dda-41da-4f10-b37a-fade93c96975 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.344291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.345162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.345447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.345722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.346919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.348408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.350015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 9/17] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:05 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:05.355227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:05.369364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:05.396020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46837b1f-ae55-4388-b047-65db34917aef None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.429006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.430138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.430421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.431000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.448993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.450453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.452766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9c5e5-e70c-4106-8605-1ae108243ebb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:23:05.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:05.471935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:05.472802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:23:05.472802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:23:05.472802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:23:05.472802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:05.472802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 9/18] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:05.478163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:05.485367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.486184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.486471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.486750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.487953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.489405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.490997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 10/19] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:05.496427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:05.534010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:05.573856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:05.576961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a28336-fc67-451c-9317-ae7b894cdced None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.592634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83481de4-6652-4b5a-83f8-1c812aac46dc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:05.609279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.609930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.610219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.610497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.611785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:05.612093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.613472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d61714-a89a-4a73-a343-c149aa04e617 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.625261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:05.625899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:05.626184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:05.627073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:05.627925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.domain.id=default, user.id=d4f3ad3806dd4f378cf1552466225045) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:05.628236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:05.629676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38ee17-7d3f-4158-be30-019d142e22d9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:05.631051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 10/20] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:05.636481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 11/21] 10.222.0.41 () {56 vars in 1322 bytes} [Mon Jun 6 20:23:05 2022] GET /identity/v3/role_assignments?user.id=d4f3ad3806dd4f378cf1552466225045&scope.domain.id=default&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 233 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:08.581817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dc5cb-2eb9-4615-b2c7-e27e6a31d827 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.582293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dc5cb-2eb9-4615-b2c7-e27e6a31d827 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.582293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dc5cb-2eb9-4615-b2c7-e27e6a31d827 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.583877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 11/22] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:08.589388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e6f5-f9e5-4f5f-a45d-544d836211c6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.590162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e6f5-f9e5-4f5f-a45d-544d836211c6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.590586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d2e6f5-f9e5-4f5f-a45d-544d836211c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.616633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-90d2e6f5-f9e5-4f5f-a45d-544d836211c6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:08.618064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-90d2e6f5-f9e5-4f5f-a45d-544d836211c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:08.707857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 12/23] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:08.742604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dc5cb-2eb9-4615-b2c7-e27e6a31d827 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:08.753763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b323c-3319-4b08-9976-629e9d4ae848 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.754326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b323c-3319-4b08-9976-629e9d4ae848 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.756580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b323c-3319-4b08-9976-629e9d4ae848 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.758102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 13/24] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:08.765831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329661e-e903-4278-9ca3-fbf7922cceed None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.768257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329661e-e903-4278-9ca3-fbf7922cceed None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.768698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329661e-e903-4278-9ca3-fbf7922cceed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.802692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c329661e-e903-4278-9ca3-fbf7922cceed None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:08.804813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c329661e-e903-4278-9ca3-fbf7922cceed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:08.930381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 14/25] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:08.932082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9ccea-7be1-4946-8b84-da06060d436e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.932806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dc5cb-2eb9-4615-b2c7-e27e6a31d827 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:08.933267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9ccea-7be1-4946-8b84-da06060d436e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.933687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9ccea-7be1-4946-8b84-da06060d436e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.936721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 15/26] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:08.942076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.942536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.942983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.960113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:08.964422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:08.975063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:08.976567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:08.987020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77f1ab-a0ca-4df3-b1eb-df88ca9c4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:08.987709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f1ab-a0ca-4df3-b1eb-df88ca9c4df4 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:08.987997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f1ab-a0ca-4df3-b1eb-df88ca9c4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:08.988272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77f1ab-a0ca-4df3-b1eb-df88ca9c4df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:08.989147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77f1ab-a0ca-4df3-b1eb-df88ca9c4df4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:08.990389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77f1ab-a0ca-4df3-b1eb-df88ca9c4df4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.054127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 16/27] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:09.055063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.165576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9544a-3338-4691-b998-088fea699fa3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.199030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 12/28] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 20:23:08 2022] POST /identity/v3/roles => generated 219 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:09.200876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415389-2879-4ca4-a7d1-fbd358bd2e2b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.201449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415389-2879-4ca4-a7d1-fbd358bd2e2b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.201991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a415389-2879-4ca4-a7d1-fbd358bd2e2b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.203646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 13/29] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:09.208760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4356b1-da7d-4ab5-8b35-79e8b9f6e455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.208760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4356b1-da7d-4ab5-8b35-79e8b9f6e455 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.208760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4356b1-da7d-4ab5-8b35-79e8b9f6e455 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.208760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4356b1-da7d-4ab5-8b35-79e8b9f6e455 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.209159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415389-2879-4ca4-a7d1-fbd358bd2e2b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.211456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:09.222492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4356b1-da7d-4ab5-8b35-79e8b9f6e455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-4e4356b1-da7d-4ab5-8b35-79e8b9f6e455 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:09.230487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:09.231317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:09.232218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 17/30] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:09.233199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.233664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.234088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.254577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:09.256159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:09.338903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 18/31] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:09.340212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.453596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c2a15-4f68-446f-8bee-cc21d7958dca None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.486821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a415389-2879-4ca4-a7d1-fbd358bd2e2b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.496223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:09.520580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.521682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.522146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.522591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.523527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:09.523861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:09.525260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.529668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1782a2-32be-4ff6-b040-e9f78eb4a801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.530342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1782a2-32be-4ff6-b040-e9f78eb4a801 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.530633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1782a2-32be-4ff6-b040-e9f78eb4a801 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.530916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1782a2-32be-4ff6-b040-e9f78eb4a801 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.531696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1782a2-32be-4ff6-b040-e9f78eb4a801 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:09.533238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1782a2-32be-4ff6-b040-e9f78eb4a801 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.537177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 19/32] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:09.538010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.599953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 14/33] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 20:23:09 2022] POST /identity/v3/roles => generated 225 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:09.601345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd33d06-7243-4965-b87b-74eefa327ae6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.601637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd33d06-7243-4965-b87b-74eefa327ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.601919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd33d06-7243-4965-b87b-74eefa327ae6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.603187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 15/34] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:09.604205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd33d06-7243-4965-b87b-74eefa327ae6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.656816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:09.659946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7c8c5-16cc-4d91-a811-12e7bbf604af None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.706303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.707026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.707313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.707592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.720675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:09.722193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.734465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd33d06-7243-4965-b87b-74eefa327ae6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-92c7067d-f972-40f7-8aae-65f37f4fb078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:23:09.741547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:09.742663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:09.743466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:23:09.743466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:23:09.743466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:23:09.743466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:09.743466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 20/35] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:09.744080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.744368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.744643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.763064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:09.764326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:09.780241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7addaace-c4a9-4efa-9b04-f104ee221792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.781387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addaace-c4a9-4efa-9b04-f104ee221792 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.781967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addaace-c4a9-4efa-9b04-f104ee221792 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.782390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7addaace-c4a9-4efa-9b04-f104ee221792 None admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.784835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addaace-c4a9-4efa-9b04-f104ee221792 None admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:09.786978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7addaace-c4a9-4efa-9b04-f104ee221792 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.788878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 16/36] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:09.790395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc12c-74e1-4758-921a-9a2af004201d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.790696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc12c-74e1-4758-921a-9a2af004201d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.790979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc12c-74e1-4758-921a-9a2af004201d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.792409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 17/37] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:09.792950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc12c-74e1-4758-921a-9a2af004201d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.804870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:09.848715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 21/38] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:09.850129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:09.914554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc12c-74e1-4758-921a-9a2af004201d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.916865 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:09.948965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.949698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.949990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.950270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.950601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77f53e-ed54-49f8-bf4a-d3b764dc10cf None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:09.950997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:09.951305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:09.953103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb0aa2-a68a-4b15-ac2f-0047075b6f97 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.964609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 18/39] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:09.965824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.966129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.966416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.983068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:09.984629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:09.985557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:09.986897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:09.987172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:09.987442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:09.988641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:09.990085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:09.991688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 22/40] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:09.992456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:10.026403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:10.051907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 19/41] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:10.052789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:10.095820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6973c2-10dd-4212-a208-c49c4af23fa9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:10.129271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:10.130045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:10.130328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:10.130599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:10.131793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:10.133252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:10.134826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 23/42] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:10.135548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:10.143999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:10.150285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3ac41-66c5-4b3f-827c-a706624b30ea None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:10.184092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:10.184761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:10.185050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:10.185332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:10.186049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:10.187510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:10.236817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b1388c-efc3-41f7-ad3b-500bc1deb170 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:10.236817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:10.264171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:10.265035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:10.265322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:10.265603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] PATH_INFO: `/v3/domains/default/users/d4f3ad3806dd4f378cf1552466225045/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:10.268240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d4f3ad3806dd4f378cf1552466225045, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:10.269683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:10.309105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 20/43] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:23:10 2022] POST /identity/v3/projects => generated 301 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:10.309105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:10.422709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ef82b2-6ec2-47c0-8f88-078224b61c94 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:10.458283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30aa05-881b-4a93-8a43-66a90e9cb3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:10.458956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30aa05-881b-4a93-8a43-66a90e9cb3a6 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:10.459243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30aa05-881b-4a93-8a43-66a90e9cb3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:10.459524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30aa05-881b-4a93-8a43-66a90e9cb3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:10.460310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30aa05-881b-4a93-8a43-66a90e9cb3a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:10.461696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c30aa05-881b-4a93-8a43-66a90e9cb3a6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:10.492726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 24/44] 10.222.0.41 () {58 vars in 1361 bytes} [Mon Jun 6 20:23:10 2022] PUT /identity/v3/domains/default/users/d4f3ad3806dd4f378cf1552466225045/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:10.493844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:10.564913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 21/45] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 20:23:10 2022] POST /identity/v3/roles => generated 223 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:10.626182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26c4c1-a260-462f-85e3-f03fff4442ee None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:10.669845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250afba-ffee-4a42-b38d-28472238d7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:10.670630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250afba-ffee-4a42-b38d-28472238d7b4 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:10.670830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250afba-ffee-4a42-b38d-28472238d7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:10.671107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250afba-ffee-4a42-b38d-28472238d7b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:10.671879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250afba-ffee-4a42-b38d-28472238d7b4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:10.673662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250afba-ffee-4a42-b38d-28472238d7b4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:10.786123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 25/46] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:23:10 2022] POST /identity/v3/projects => generated 312 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:12.733800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828591d-bfe8-4b65-8468-40a9a5d91269 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:12.734297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828591d-bfe8-4b65-8468-40a9a5d91269 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:12.734378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828591d-bfe8-4b65-8468-40a9a5d91269 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:12.736364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 22/47] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:12.741775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f18f7d-2488-44e9-bd45-7af0abd14f29 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:12.742188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f18f7d-2488-44e9-bd45-7af0abd14f29 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:12.742350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f18f7d-2488-44e9-bd45-7af0abd14f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:12.759473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-64f18f7d-2488-44e9-bd45-7af0abd14f29 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:12.761041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-64f18f7d-2488-44e9-bd45-7af0abd14f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:12.771606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:12.825988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 26/48] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:12.849275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828591d-bfe8-4b65-8468-40a9a5d91269 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:12.858508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:12.955536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828591d-bfe8-4b65-8468-40a9a5d91269 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:12.988778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:12.989494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:12.989781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:12.990061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.001407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:13.008928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f9e0a-0d63-4804-82ff-e6e21872408e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.009223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f9e0a-0d63-4804-82ff-e6e21872408e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.009505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f9e0a-0d63-4804-82ff-e6e21872408e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:13.010182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:13.011049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:13.011840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:13.012401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 27/49] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:23:13.012476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 23/50] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:13.016082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.016369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.016699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:13.016815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.034326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:13.035857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:13.080599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:13.103981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 28/51] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:13.115566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3cba1-004f-4ddf-866d-57f2ed56211f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:13.127687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:13.148807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:13.149467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.149752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.150031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.150779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:13.151087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:13.152505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:13.165753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 24/52] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:13.171123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:13.201803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:13.224784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263d73e-9fb5-4ae0-8e61-432224051c37 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:13.261435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:13.262146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.262435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.262716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.263928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:13.265392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:13.267556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 29/53] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:13.269378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da38184-7224-40f4-b3d9-50b44413f015 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:13.272944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:13.296491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:13.304512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:13.305367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.305648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.305918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.307107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:13.308594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:13.314465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 25/54] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:13 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:13.315372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:13.370560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960f8d3-6b96-4454-94c8-b5776519edcc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:13.403500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf6c35-9a3f-46ad-ae21-200846711b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:13.404185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf6c35-9a3f-46ad-ae21-200846711b99 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.404472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf6c35-9a3f-46ad-ae21-200846711b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.404758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf6c35-9a3f-46ad-ae21-200846711b99 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.405459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf6c35-9a3f-46ad-ae21-200846711b99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:13.406905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf6c35-9a3f-46ad-ae21-200846711b99 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:13.414399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f8c3e-1b63-4359-b8b6-83d40b21a83d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:13.421141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:13.447327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:13.448146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.448485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.448733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.449896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:13.451328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:13.452968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 26/55] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:13.459064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:13.518205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:13.542291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 30/56] 10.222.0.41 () {60 vars in 1140 bytes} [Mon Jun 6 20:23:13 2022] POST /identity/v3/projects => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:23:13.557057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7afcaf-f142-4ad9-af25-8df1ac322c1b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:13.594301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:13.594985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:13.595270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:13.595545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:13.596425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.domain.id=default, user.id=d4f3ad3806dd4f378cf1552466225045) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:13.596736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:13.598193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c8bae-7d5b-4f71-acee-bdb95c6dc43c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:13.607951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 27/57] 10.222.0.41 () {56 vars in 1322 bytes} [Mon Jun 6 20:23:13 2022] GET /identity/v3/role_assignments?user.id=d4f3ad3806dd4f378cf1552466225045&scope.domain.id=default&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:15.800602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b813a-df7f-4729-8897-a7fcbf74d4fe None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:15.801109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b813a-df7f-4729-8897-a7fcbf74d4fe None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:15.801221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b813a-df7f-4729-8897-a7fcbf74d4fe None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:15.802563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 31/58] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:15.807750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:15.808107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:15.808315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:15.824537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:15.825703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:15.889254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 28/59] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:15.912963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b813a-df7f-4729-8897-a7fcbf74d4fe None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:16.009385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b813a-df7f-4729-8897-a7fcbf74d4fe None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:16.041492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98789c3b-e13f-44f0-8c43-40b419ce0d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:16.042274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98789c3b-e13f-44f0-8c43-40b419ce0d24 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:16.042562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98789c3b-e13f-44f0-8c43-40b419ce0d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:16.042842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98789c3b-e13f-44f0-8c43-40b419ce0d24 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:16.044031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98789c3b-e13f-44f0-8c43-40b419ce0d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:16.045493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98789c3b-e13f-44f0-8c43-40b419ce0d24 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:16.047078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 32/60] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:16.053046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:16.107786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:16.147296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca0bfd-4833-474a-a1db-8760f46219e7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:16.179552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14975c14-e59c-4ccc-87fb-656d3d11d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:16.180235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975c14-e59c-4ccc-87fb-656d3d11d57e None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:16.180521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975c14-e59c-4ccc-87fb-656d3d11d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:16.180798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14975c14-e59c-4ccc-87fb-656d3d11d57e None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:16.181505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975c14-e59c-4ccc-87fb-656d3d11d57e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:16.182946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14975c14-e59c-4ccc-87fb-656d3d11d57e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:16.289788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 29/61] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:16 2022] POST /identity/v3/users => generated 275 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:19.610548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c00ff-eff8-4498-b7a6-101eb72fb5f6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:19.611028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c00ff-eff8-4498-b7a6-101eb72fb5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:19.611028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0c00ff-eff8-4498-b7a6-101eb72fb5f6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:19.612422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 33/62] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:19.624594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:19.625127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:19.625127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:19.650702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:19.651945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:19.661344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cb6d1-c781-449e-b4b8-71ddce495fc4 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:19.661978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cb6d1-c781-449e-b4b8-71ddce495fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:19.662449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600cb6d1-c781-449e-b4b8-71ddce495fc4 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:19.664339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 34/63] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:19.677458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:19.678060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:19.678534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:19.702865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:19.705767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:19.760837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 30/64] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:19.785461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:19.822848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:19.832316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 35/65] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:19.852031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:19.900657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:19.947530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97615db9-30fc-41d4-8333-a16e427d1eea None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.022103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d729e4-2283-45e9-bbb4-05624b50a548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.023302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d729e4-2283-45e9-bbb4-05624b50a548 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.023835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d729e4-2283-45e9-bbb4-05624b50a548 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.024315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d729e4-2283-45e9-bbb4-05624b50a548 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.040112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d729e4-2283-45e9-bbb4-05624b50a548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f9d729e4-2283-45e9-bbb4-05624b50a548 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:20.051878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:20.052710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:20.053467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:20.054026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779174cc-978b-4cb1-b6de-b964f78f6829 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.061941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 31/66] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:20.064274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9a6b3-a1ba-425f-b5c0-8832588d9aba None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.064719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9a6b3-a1ba-425f-b5c0-8832588d9aba None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.065147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9a6b3-a1ba-425f-b5c0-8832588d9aba None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.067863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 32/67] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:20.069898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9a6b3-a1ba-425f-b5c0-8832588d9aba None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.101885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc83e1-413b-4728-91de-f10848ad5045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.103190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc83e1-413b-4728-91de-f10848ad5045 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.103813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc83e1-413b-4728-91de-f10848ad5045 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.104238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc83e1-413b-4728-91de-f10848ad5045 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.118839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc83e1-413b-4728-91de-f10848ad5045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-f1bc83e1-413b-4728-91de-f10848ad5045 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:20.129611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:20.130700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:20.131651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:20.134827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 36/68] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:20.136269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781606d-3161-463d-a438-c16408325dd1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.136717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781606d-3161-463d-a438-c16408325dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.137125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781606d-3161-463d-a438-c16408325dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.155046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6781606d-3161-463d-a438-c16408325dd1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:20.156206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6781606d-3161-463d-a438-c16408325dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:20.187832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9a6b3-a1ba-425f-b5c0-8832588d9aba None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.190897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.207727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.223225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.223963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.224213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.224494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.225651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:20.225651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:20.226906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 37/69] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:20.226906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781606d-3161-463d-a438-c16408325dd1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.237204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef931d2a-645d-42a4-9cb2-a4e6402aedc6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:20.250446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 33/70] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:20.252400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96392d-3583-4f0c-a547-1157eafad150 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.252698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96392d-3583-4f0c-a547-1157eafad150 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.252981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96392d-3583-4f0c-a547-1157eafad150 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.254825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 34/71] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:20.255764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96392d-3583-4f0c-a547-1157eafad150 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.342834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781606d-3161-463d-a438-c16408325dd1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.364721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96392d-3583-4f0c-a547-1157eafad150 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.375068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.375756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.376580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.376857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.377594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:20.377895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:20.379278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:20.387521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 38/72] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:20.388167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.399065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c7f6d7-5476-4fa7-a3a6-33fabd8f77cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.399895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7f6d7-5476-4fa7-a3a6-33fabd8f77cc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.400188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7f6d7-5476-4fa7-a3a6-33fabd8f77cc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.400466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c7f6d7-5476-4fa7-a3a6-33fabd8f77cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.410952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c7f6d7-5476-4fa7-a3a6-33fabd8f77cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c5c7f6d7-5476-4fa7-a3a6-33fabd8f77cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:20.418863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:20.419758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:20.420490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 35/73] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:20.421281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a63005-1b0f-4f6a-8f31-5a81bee1f465 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.421571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a63005-1b0f-4f6a-8f31-5a81bee1f465 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.421913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a63005-1b0f-4f6a-8f31-5a81bee1f465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.436139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.437758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e8a63005-1b0f-4f6a-8f31-5a81bee1f465 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:20.439488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e8a63005-1b0f-4f6a-8f31-5a81bee1f465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:20.459676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.488320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99a9e0-04c3-4aae-ba5f-25a72f9f0cac None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.504096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 36/74] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:20.505446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77c1c-d9f1-46f7-9911-c4df750a8693 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.505737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77c1c-d9f1-46f7-9911-c4df750a8693 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.506015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e77c1c-d9f1-46f7-9911-c4df750a8693 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.511475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 37/75] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:20.511475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77c1c-d9f1-46f7-9911-c4df750a8693 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.526732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.527619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.527941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.528228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.541610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:20.543080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:20.544765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 39/76] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:20.545534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.611011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e77c1c-d9f1-46f7-9911-c4df750a8693 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.641463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5420b-20ea-4f8d-a28d-b7151e985948 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.643760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.645352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655b2ed-deeb-4d1b-922b-98c668558a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.646220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655b2ed-deeb-4d1b-922b-98c668558a32 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.646503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655b2ed-deeb-4d1b-922b-98c668558a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.646783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655b2ed-deeb-4d1b-922b-98c668558a32 None admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.647999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655b2ed-deeb-4d1b-922b-98c668558a32 None admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:20.649453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655b2ed-deeb-4d1b-922b-98c668558a32 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:20.651069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 38/77] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:20.652291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.652589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.652869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.670552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:20.671486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:20.674639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.675325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.675609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.675915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.676652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:20.676956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:20.678331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:20.686056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 40/78] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:20.686056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.692464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.737715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 39/79] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:20.738535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.784923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573410e3-8ba1-4dea-b8df-a154f3dc4604 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.817727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.818507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.818787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.819066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.828758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:20.836312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:20.837149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:20.837899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 41/80] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:20.838507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:20.850875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7b41d-d889-4f2d-97a2-4421a5ecb163 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.864554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.886303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.886953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.887241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.887525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:20.911375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:20.940844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41bb59-812f-4288-a723-c3b5dbbf7eb1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:20.954907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:20.956399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:20.972725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:20.973363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:20.973645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:20.973930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.039057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.040588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.053202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 40/81] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:21.054239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.070848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.107390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.141193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 42/82] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:21.145827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.150363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f4791-0bc9-4b6d-aa9a-5c7d662a1e1a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.183996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.184859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.185135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.185415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.186594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.188076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.190222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 41/83] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.191023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.242671 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccd592-2234-467a-a318-466f63c2bcfe None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.268351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.276810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.277606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.287026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.287026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.287026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.287026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.287026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 43/84] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.287026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.291407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576fab0-04a6-47d6-9d69-aca151a9b336 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.314152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.325133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.325795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.326081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.329327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.329327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:21.329327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.329327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.336937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 42/85] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.337720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.383629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9b915e-e66d-402f-b22a-a8cb66d975f0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.417121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.417969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.418258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.418537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.419733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.421174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.422878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 44/86] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.423738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.437411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3664-7355-4639-8278-82967a2a8f7b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.469809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.471186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.471472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.471779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.472979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.474414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.476139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 43/87] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.480470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.483258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.520105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a3f09-ce13-4abf-aa0a-25b14699ed71 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.531456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.554266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.554900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.555188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.555465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.577473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6618d678-961f-4a46-b8d6-34143c1b59ae None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.609982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.610637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.610926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.611207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.611950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.613404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.620711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.622166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.678818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.715550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 45/88] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:21.716447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.727078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 44/89] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:21 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:21.727078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.826782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deaffa9-8630-49fa-a7dc-ab93314e13f0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.835034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bef6a8-8bbf-4e51-be68-743aa2a87442 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:21.845316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:21.864039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.864922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.865213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.865489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.866776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.868291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.874468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 46/90] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.874468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.878882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:21.879631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:21.879956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:21.880235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:21.881087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=22fb0cc1f4074e9e9e9005557751064f) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:21.884235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:21.884235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:21.889350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 45/91] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/role_assignments?user.id=22fb0cc1f4074e9e9e9005557751064f&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:21.890606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:21.923810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:22.015933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3137c44-f714-4e45-94f7-93d552a3e339 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:22.033041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3087e8c-ad17-40da-9926-ffb77e410fed None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:22.084140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:22.084856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:22.085099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:22.085429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:22.086217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=d4f3ad3806dd4f378cf1552466225045) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:22.086538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:22.086955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:22.087981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:22.088158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:22.088320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:22.088539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:22.089777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:22.091248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:22.093069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 46/92] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:22.093876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:22.095844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 47/93] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:21 2022] GET /identity/v3/role_assignments?user.id=d4f3ad3806dd4f378cf1552466225045&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:22.100495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:22.152724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:22.242128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:22.252518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aefe55d-ea76-43dd-9e28-e8c3ac78bbe0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:22.266203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b15fd-00a0-4e9f-8819-7adf7f511071 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:22.314030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:22.314688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:22.314971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:22.315248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:22.354198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:22.356021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:22.356311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:22.356592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:22.358249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] RBAC: Adding query filter params (role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=22fb0cc1f4074e9e9e9005557751064f) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:22.358779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:22.361670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:22.377784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 48/94] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:22 2022] GET /identity/v3/role_assignments?user.id=22fb0cc1f4074e9e9e9005557751064f&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:22.429878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:22.431371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:22.490072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:22.567128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 47/95] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:22.572461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:22.714800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ca94c-515e-4106-8983-35e0f9c187c1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:22.763109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fa5e8-cfd9-4852-8a4f-91592337e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:22.764358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fa5e8-cfd9-4852-8a4f-91592337e869 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:22.764849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fa5e8-cfd9-4852-8a4f-91592337e869 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:22.765325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fa5e8-cfd9-4852-8a4f-91592337e869 None admin] PATH_INFO: `/v3/projects/72004884fa2849369e75500d5f62a258` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:22.767383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fa5e8-cfd9-4852-8a4f-91592337e869 None admin] RBAC: Authorizing `identity:get_project(project_id=72004884fa2849369e75500d5f62a258)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:22.769125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fa5e8-cfd9-4852-8a4f-91592337e869 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:22.771125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 49/96] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:22 2022] GET /identity/v3/projects/72004884fa2849369e75500d5f62a258 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:22.780817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:22.929129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009e90c-e442-4865-ba85-cefca16f9eb4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:22.981742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:22.982400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:22.982685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:22.982963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:22.983824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=72004884fa2849369e75500d5f62a258, user.id=22fb0cc1f4074e9e9e9005557751064f) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:22.984136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:22.986126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b27b35-caa4-4f7c-a357-a1d82a16c8a7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:22.995819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 48/97] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:22 2022] GET /identity/v3/role_assignments?user.id=22fb0cc1f4074e9e9e9005557751064f&scope.project.id=72004884fa2849369e75500d5f62a258&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:24.760599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf930682-44eb-46ea-8790-474e9db7feed None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:24.761716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf930682-44eb-46ea-8790-474e9db7feed None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:24.762214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf930682-44eb-46ea-8790-474e9db7feed None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:24.763836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 50/98] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:24.770116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208abdd-9a8f-4a4e-a41c-7e90605960f7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:24.771060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208abdd-9a8f-4a4e-a41c-7e90605960f7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:24.771544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208abdd-9a8f-4a4e-a41c-7e90605960f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:24.782418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:24.796668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3208abdd-9a8f-4a4e-a41c-7e90605960f7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:24.803655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3208abdd-9a8f-4a4e-a41c-7e90605960f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:24.878915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 49/99] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:24.918857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf930682-44eb-46ea-8790-474e9db7feed None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:24.952281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:24.976418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd6b1b-bd96-4e06-b9ee-d2f70bd71288 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:24.976743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd6b1b-bd96-4e06-b9ee-d2f70bd71288 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:24.976987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd6b1b-bd96-4e06-b9ee-d2f70bd71288 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:24.978368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 50/100] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:24.983959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85133e24-f927-4eb1-b312-95232a391e92 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:24.984251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85133e24-f927-4eb1-b312-95232a391e92 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:24.984535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85133e24-f927-4eb1-b312-95232a391e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.014468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-85133e24-f927-4eb1-b312-95232a391e92 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:25.015573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-85133e24-f927-4eb1-b312-95232a391e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:25.110917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf930682-44eb-46ea-8790-474e9db7feed None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.159536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 51/101] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:25.161170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c5b26-a706-463a-982b-e8d99cbea352 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.161460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c5b26-a706-463a-982b-e8d99cbea352 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.161738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c5b26-a706-463a-982b-e8d99cbea352 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.162997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 52/102] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:25.172465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.173282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.173575 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.187581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:25.189027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.189324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.189598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.205364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:25.206534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:25.213281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:25.226331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:25.227255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:25.228060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 51/103] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:25.228729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.339730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:25.340980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 53/104] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:25.347941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.408352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:25.425425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed177-6e7a-405a-8f31-f1ef72334a1c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.476214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:25.477035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.477330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.477609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.497712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:25.508302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:25.509160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 52/105] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:25.510318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.565884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba221ed3-4a15-4720-ac45-69d6a25e0a3c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.630123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:25.630720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.631006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.631293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.632171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:25.632486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:25.635051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:25.647637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 54/106] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:25.648716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.676377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d8218-3d65-486d-a9ba-2e9f43032daf None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.708340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:25.738018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:25.738861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.739148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.739416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.744907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:25.746591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:25.748595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 53/107] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:25.749509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.773307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414316-bd04-40c1-af6c-18c9156dec50 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.784440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:25.828639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:25.829693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.831187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.831765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.832786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:25.833278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:25.834924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:25.845416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 55/108] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:25.846615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.906832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c716f-4d03-4ef8-b66b-27b19499fdde None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.944674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74f5ec-7a9b-45b9-ae23-7c2fe4f1c7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:25.945957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f5ec-7a9b-45b9-ae23-7c2fe4f1c7df None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.946432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f5ec-7a9b-45b9-ae23-7c2fe4f1c7df None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.946957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74f5ec-7a9b-45b9-ae23-7c2fe4f1c7df None admin] PATH_INFO: `/v3/users/f54c2f5b0547465cb784fc695cf6e294` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.962509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f5ec-7a9b-45b9-ae23-7c2fe4f1c7df None admin] RBAC: Authorizing `identity:get_user(user_id=f54c2f5b0547465cb784fc695cf6e294)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:25.964344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74f5ec-7a9b-45b9-ae23-7c2fe4f1c7df None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:25.966565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 54/109] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:25 2022] GET /identity/v3/users/f54c2f5b0547465cb784fc695cf6e294 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:25.968064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06084398-40b4-4546-a3c7-d763600dde70 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:25.968196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdeec4-6fda-4d5e-930a-74e13326f74b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:25.968781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06084398-40b4-4546-a3c7-d763600dde70 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:25.969253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06084398-40b4-4546-a3c7-d763600dde70 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:25.971528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 55/110] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:25.972451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06084398-40b4-4546-a3c7-d763600dde70 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:25.992581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.014311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.015028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.015317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.015598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.016343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:26.019516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:26.060808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.088903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 56/111] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:23:25 2022] POST /identity/v3/projects => generated 302 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:26.090612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.094127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06084398-40b4-4546-a3c7-d763600dde70 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.149964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac58411e-a227-4043-a031-be4810762210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.151009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58411e-a227-4043-a031-be4810762210 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.151410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58411e-a227-4043-a031-be4810762210 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.151827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac58411e-a227-4043-a031-be4810762210 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.153799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58411e-a227-4043-a031-be4810762210 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:26.155274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac58411e-a227-4043-a031-be4810762210 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:26.158169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 56/112] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:25 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:26.159535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3aee0-8e93-40a6-86a3-08940e76b8c7 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.160238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3aee0-8e93-40a6-86a3-08940e76b8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.160622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3aee0-8e93-40a6-86a3-08940e76b8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.183295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-e0e3aee0-8e93-40a6-86a3-08940e76b8c7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:26.186228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e0e3aee0-8e93-40a6-86a3-08940e76b8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:26.232059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bffb60c-6533-4ebf-a50f-62313cd1ae6f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.269420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.272675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.272675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.272675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.278694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 57/113] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:26.279995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11282284-6668-423f-bc9f-177a02077f46 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.280297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11282284-6668-423f-bc9f-177a02077f46 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.280584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11282284-6668-423f-bc9f-177a02077f46 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.283931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 58/114] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:26.283931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11282284-6668-423f-bc9f-177a02077f46 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.330954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.361565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:26.363067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:26.372615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.396689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11282284-6668-423f-bc9f-177a02077f46 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.431689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d76049f-84e1-4e37-bbd7-99db5b7b0852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.432488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76049f-84e1-4e37-bbd7-99db5b7b0852 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.432781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76049f-84e1-4e37-bbd7-99db5b7b0852 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.433063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d76049f-84e1-4e37-bbd7-99db5b7b0852 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.434490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76049f-84e1-4e37-bbd7-99db5b7b0852 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:26.435989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d76049f-84e1-4e37-bbd7-99db5b7b0852 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:26.437987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 59/115] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:26.439712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860970a2-7aec-4ca8-910d-7d8a574c3644 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.440002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860970a2-7aec-4ca8-910d-7d8a574c3644 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.440284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860970a2-7aec-4ca8-910d-7d8a574c3644 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.441669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 60/116] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:26.442577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.442877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.443567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.463201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:26.464414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:26.474194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 57/117] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:26.475294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.535250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 61/118] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:26.536006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.547853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.581172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd88e3-b40e-4a7c-b794-340dd25ac937 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.610090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.616965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaa199-7f0f-4d68-b556-3e52830271d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.621854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa199-7f0f-4d68-b556-3e52830271d9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.621854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa199-7f0f-4d68-b556-3e52830271d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.621854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa199-7f0f-4d68-b556-3e52830271d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.629258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eaa199-7f0f-4d68-b556-3e52830271d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b1eaa199-7f0f-4d68-b556-3e52830271d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:26.636887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:26.637925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:26.638716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 58/119] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:26.639353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.639564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.639873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.655418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40648e-045e-4568-a4b0-654fa7b708b4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.656656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:26.657886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:26.691966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.692786 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.693076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.693358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1/users/22fb0cc1f4074e9e9e9005557751064f/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.695350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7079597609471ba368aba9a176b5f1, user_id=22fb0cc1f4074e9e9e9005557751064f, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:26.696843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:26.726866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 59/120] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:26.727736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.764106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 62/121] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:26 2022] PUT /identity/v3/projects/2a7079597609471ba368aba9a176b5f1/users/22fb0cc1f4074e9e9e9005557751064f/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:26.765122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.794855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.840047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e850ea-3b4a-4264-8e7d-513b30a45908 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.879263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.880390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.880717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.881094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.882526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:26.884154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:26.886382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 60/122] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:26.892160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:26.893978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66e458-8c68-46f9-b240-d32708c6a23d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:26.926066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:26.936193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:26.937498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:26.937935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:26.938448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:26.952634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:26.962968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:26.963937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:26.964719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:26.968438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 63/123] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:26.974624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.008729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5cebf7-fbbf-4d18-b4bc-fb1ae302ec2a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.046674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:27.060866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.061529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.063023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.063322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.064109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:27.064414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.065786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.074125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 61/124] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.074872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.113280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357cc80-2172-4ee5-8455-1343242f49f9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.149745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.150552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.150835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.151516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.163195 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:27.171045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:27.171934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:27.173681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 64/125] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:27.174544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.198717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848ca81-10c8-42e4-ba3c-a035358caead None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.225478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:27.252470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.253279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.253677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.254131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.255109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] RBAC: Adding query filter params (role.id=dd5b92e530a949509eb3772b4a01aac0, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=f54c2f5b0547465cb784fc695cf6e294) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:27.255537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.257595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.265629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 62/126] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/role_assignments?user.id=f54c2f5b0547465cb784fc695cf6e294&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=dd5b92e530a949509eb3772b4a01aac0 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.272033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.307683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43cc11-be57-4396-8c5a-cf7334fa57d8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.376643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:27.427599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.428346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.428639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.428918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.429686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:27.429999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.431451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.449531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595cb36-b8bd-47ea-8ee9-a2d7ba44df73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.453440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 65/127] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.456434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.512763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.513995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.514484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.514964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.517196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.519983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.522169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 63/128] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.523157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.599245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df53d2a-1a56-4f2b-a7a9-3bd1ef0149ff None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.617427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:27.676176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016459dd-8da3-4d6f-9a01-3ac2aed66685 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.689373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.690428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.690909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.691408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.692377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:27.692879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.694517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.703705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:27.711519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 66/129] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.712634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.715055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.716296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.716601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.716871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.719258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.721367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.723023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 64/130] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.723901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.893816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a54a8-e8cc-4d0d-ae5d-de3f35b3d9af None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.923336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df21d68-642a-4d31-b924-c076e5f6a96b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:27.948358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.949876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.950373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.950931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.953581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.955885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:27.961687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 67/131] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:27.962767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:27.988185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:27.989469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:27.989947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:27.990424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:27.992721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:27.995626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:28.002688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 65/132] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:28.003713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:28.045885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:28.116076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:28.152708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d737f-b0c1-4b14-804f-9bb1f6ff7190 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:28.192885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:28.194118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.194708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.195228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1/users/d4f3ad3806dd4f378cf1552466225045/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.197676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7079597609471ba368aba9a176b5f1, user_id=d4f3ad3806dd4f378cf1552466225045, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:28.197800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12251085-5c76-4269-94b8-ca7b7c9301e8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:28.199727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:28.245658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:28.246538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.246873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.247156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] PATH_INFO: `/v3/projects/72004884fa2849369e75500d5f62a258` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.248441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] RBAC: Authorizing `identity:get_project(project_id=72004884fa2849369e75500d5f62a258)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:28.250004 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:28.252710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 68/133] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:27 2022] GET /identity/v3/projects/72004884fa2849369e75500d5f62a258 => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:28.256623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:28.318417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 66/134] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:28 2022] PUT /identity/v3/projects/2a7079597609471ba368aba9a176b5f1/users/d4f3ad3806dd4f378cf1552466225045/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:28.319554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:28.368656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf8a61-5458-4c4f-907d-224b95dbeadb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:28.378853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:28.408764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:28.410214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.410646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.411066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.412503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:28.414225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:28.416468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 69/135] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:28 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:28.421944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:28.494234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712a696-207a-4f38-97aa-3e168609f13b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:28.516138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:28.544742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff8536-c735-4a04-a58f-903740f6e4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:28.545663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff8536-c735-4a04-a58f-903740f6e4b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.545956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff8536-c735-4a04-a58f-903740f6e4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.546244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff8536-c735-4a04-a58f-903740f6e4b4 None admin] PATH_INFO: `/v3/projects/72004884fa2849369e75500d5f62a258/users/22fb0cc1f4074e9e9e9005557751064f/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.556557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff8536-c735-4a04-a58f-903740f6e4b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=72004884fa2849369e75500d5f62a258, user_id=22fb0cc1f4074e9e9e9005557751064f, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:28.558059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff8536-c735-4a04-a58f-903740f6e4b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:28.603808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96152f-94c3-4a36-8cc1-916427a67e3d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:28.645453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:28.646999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.647285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.647563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1/users/22fb0cc1f4074e9e9e9005557751064f/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.650291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7079597609471ba368aba9a176b5f1, user_id=22fb0cc1f4074e9e9e9005557751064f, role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:28.651084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 67/136] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:28 2022] PUT /identity/v3/projects/72004884fa2849369e75500d5f62a258/users/22fb0cc1f4074e9e9e9005557751064f/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:28.657362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285e56b-c03e-42e7-bd23-b56bf37ed220 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.657362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285e56b-c03e-42e7-bd23-b56bf37ed220 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.657362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285e56b-c03e-42e7-bd23-b56bf37ed220 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.657657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:28.657728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 68/137] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:28.661627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:28.661627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:28.661627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:28.683234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:28.684532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:28.745593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 70/138] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:28 2022] PUT /identity/v3/projects/2a7079597609471ba368aba9a176b5f1/users/22fb0cc1f4074e9e9e9005557751064f/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:28.768659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 69/139] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:28.796058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:28.869595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:28.955164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe29671-5875-40fe-82a0-d0d6419bbf5b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:29.005919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac84c3a-200a-44dd-8fa2-1828e527a9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:29.006743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac84c3a-200a-44dd-8fa2-1828e527a9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:29.007034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac84c3a-200a-44dd-8fa2-1828e527a9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:29.007313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac84c3a-200a-44dd-8fa2-1828e527a9c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:29.008581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac84c3a-200a-44dd-8fa2-1828e527a9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:29.010094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac84c3a-200a-44dd-8fa2-1828e527a9c6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:29.011820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 71/140] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:29.018122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:29.161780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c75920e-1662-467e-acce-8bbcf8200811 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:29.175621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:29.207807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdc089d-c861-4325-a6fe-baea94f72489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:29.208404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc089d-c861-4325-a6fe-baea94f72489 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:29.208687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc089d-c861-4325-a6fe-baea94f72489 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:29.208965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdc089d-c861-4325-a6fe-baea94f72489 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:29.209677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc089d-c861-4325-a6fe-baea94f72489 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:29.211108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdc089d-c861-4325-a6fe-baea94f72489 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:29.330623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 70/141] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:29 2022] POST /identity/v3/users => generated 283 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:29.440854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5590d8-2bb4-47bb-8a2b-9e6c487d59f1 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:29.441500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5590d8-2bb4-47bb-8a2b-9e6c487d59f1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:29.441992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5590d8-2bb4-47bb-8a2b-9e6c487d59f1 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:29.443579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 72/142] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:29.449107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:29.449398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:29.449683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:29.472880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:29.474099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:29.570332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 71/143] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:29.605782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5590d8-2bb4-47bb-8a2b-9e6c487d59f1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:29.758222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5590d8-2bb4-47bb-8a2b-9e6c487d59f1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:29.764824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:29.806044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:29.807206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:29.807745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:29.808222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:29.824960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:29.833072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:29.833908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:29.834655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:29.836990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 73/144] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:29.844786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:29.957153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:29.964105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfb807-eaf2-4bf8-a970-8c3e7eb92d27 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:30.003926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:30.004607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.004897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.005179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.005950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:30.006267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:30.007705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:30.016710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 72/145] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:30.022249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:30.176208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e594fbc-465c-4c05-8a8a-fb4b00949ebb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:30.223396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ad849-8cca-4c63-9e29-5bd7aeea1a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:30.225077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad849-8cca-4c63-9e29-5bd7aeea1a81 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.225554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad849-8cca-4c63-9e29-5bd7aeea1a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.225985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ad849-8cca-4c63-9e29-5bd7aeea1a81 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.228422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ad849-8cca-4c63-9e29-5bd7aeea1a81 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:30.229835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ad849-8cca-4c63-9e29-5bd7aeea1a81 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:30.231719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 74/146] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:30 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:30.239689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:30.325574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2681345-1857-44ab-be73-7cef502767e5 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.325881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2681345-1857-44ab-be73-7cef502767e5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.326047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2681345-1857-44ab-be73-7cef502767e5 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.327428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 75/147] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:30.335182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.335472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.335781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.372451 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc78ca2-68e7-4466-8656-a29f1bca0805 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:30.377426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:30.378700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:30.430800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:30.431831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.432359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.432857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.435546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:30.464882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:30.512988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:30.514799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:30.527038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 76/148] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:30.572508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:30.669344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 73/149] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:30.677726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:30.747904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49c4be7-0c90-4acd-93bf-a0bb7da41216 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:30.763692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:30.821930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:30.822997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.823424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.823867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.841420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:30.845588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b383659-2a5c-4ae8-b58c-3b1ddea29e7c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:30.850209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:30.851052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:30.851823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:30.854284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 77/150] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:30.877315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:30.897382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:30.898350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:30.898709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:30.899060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:30.900479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:30.902028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:30.903864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 74/151] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:30 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:30.912590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:30.925986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:31.086597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402f5bd2-9ac8-46c7-b18b-a00dab92e9f6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:31.094157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3b9b2-f4f3-4d23-9d62-3c64ead53303 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:31.132777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:31.164651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:31.165321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:31.165614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:31.165892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:31.166734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=22fb0cc1f4074e9e9e9005557751064f) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:31.167045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:31.168516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:31.171607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:31.172299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:31.172614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:31.172898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:31.173678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:31.173992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:31.175382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:31.187363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 75/152] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:30 2022] GET /identity/v3/role_assignments?user.id=22fb0cc1f4074e9e9e9005557751064f&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:31.189617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 78/153] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:31.195235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:31.351814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6cbe39-a8a0-4c8f-ba4d-68330996d721 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:31.405862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:31.406767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:31.407638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:31.407959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] PATH_INFO: `/v3/users/f54c2f5b0547465cb784fc695cf6e294` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:31.412509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] RBAC: Authorizing `identity:get_user(user_id=f54c2f5b0547465cb784fc695cf6e294)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:31.414047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:31.416211 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 76/154] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:31 2022] GET /identity/v3/users/f54c2f5b0547465cb784fc695cf6e294 => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:31.421724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:31.550166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:31.568316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f68783-af55-42ff-9900-bbb38da860f8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:31.611580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b97c5-dc96-4ea4-a98a-436739b11c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:31.612490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b97c5-dc96-4ea4-a98a-436739b11c36 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:31.612771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b97c5-dc96-4ea4-a98a-436739b11c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:31.613042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b97c5-dc96-4ea4-a98a-436739b11c36 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:31.614993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b97c5-dc96-4ea4-a98a-436739b11c36 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:31.616497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b97c5-dc96-4ea4-a98a-436739b11c36 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:31.618201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 79/155] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:31 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:31.624101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:31.701339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:31.746345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dc70a-34c5-48b5-bd17-02307fe4a501 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:31.785695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23a7c1-54e7-4e41-98df-88fda6b70280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:31.786623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a7c1-54e7-4e41-98df-88fda6b70280 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:31.786912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a7c1-54e7-4e41-98df-88fda6b70280 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:31.787186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23a7c1-54e7-4e41-98df-88fda6b70280 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1/users/f54c2f5b0547465cb784fc695cf6e294/roles/dd5b92e530a949509eb3772b4a01aac0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:31.793254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23a7c1-54e7-4e41-98df-88fda6b70280 None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7079597609471ba368aba9a176b5f1, user_id=f54c2f5b0547465cb784fc695cf6e294, role_id=dd5b92e530a949509eb3772b4a01aac0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:31.794727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23a7c1-54e7-4e41-98df-88fda6b70280 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:31.878921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 77/156] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:31 2022] PUT /identity/v3/projects/2a7079597609471ba368aba9a176b5f1/users/f54c2f5b0547465cb784fc695cf6e294/roles/dd5b92e530a949509eb3772b4a01aac0 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:32.173482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649250a-e004-4fe6-bf0b-27c228693bc0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.173985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649250a-e004-4fe6-bf0b-27c228693bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.173985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649250a-e004-4fe6-bf0b-27c228693bc0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.175639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 80/157] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:32.187467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9d6b6-1942-4002-820c-e1bc34eb2e03 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.187779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9d6b6-1942-4002-820c-e1bc34eb2e03 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.188063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9d6b6-1942-4002-820c-e1bc34eb2e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.213315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-aac9d6b6-1942-4002-820c-e1bc34eb2e03 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:32.214540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aac9d6b6-1942-4002-820c-e1bc34eb2e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:32.299677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 78/158] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:32.338974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649250a-e004-4fe6-bf0b-27c228693bc0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:32.416832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a150e07-6fc5-44a0-b246-3de8d7a55d1a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.417357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a150e07-6fc5-44a0-b246-3de8d7a55d1a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.417431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a150e07-6fc5-44a0-b246-3de8d7a55d1a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.418759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 79/159] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:32.425862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.426173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.426451 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.455615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:32.456828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:32.517400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649250a-e004-4fe6-bf0b-27c228693bc0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:32.546175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:32.571347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:32.572176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.572462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.572739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.588851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:32.605643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:32.609057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:32.609884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:32.611077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 80/160] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:32.611148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 81/161] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:32.614495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:32.657840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:32.683401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:32.832340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702ad297-9199-4dc2-a6bf-47c1b6e8febc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:32.849903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2521bf-7a05-45d0-975f-cbd9bc07fda5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:32.888997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:32.889813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.890099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.890378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.908916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:32.921368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:32.922238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:32.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 82/162] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:32.930467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:32.953070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:32.953681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:32.954508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:32.954787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:32.955554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:32.955885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:32.957280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:32.973655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 81/163] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:32.979769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:33.051698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:33.096022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:33.146010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c0871-33ac-4919-b173-21df91d26371 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:33.168010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674a234-9be1-4269-b4ae-d9a227c58f13 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:33.222956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccbaa15-17dd-4494-bffa-0a01158b6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:33.223874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbaa15-17dd-4494-bffa-0a01158b6d08 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:33.224164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbaa15-17dd-4494-bffa-0a01158b6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:33.224442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccbaa15-17dd-4494-bffa-0a01158b6d08 None admin] PATH_INFO: `/v3/users/d4f3ad3806dd4f378cf1552466225045` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:33.228226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbaa15-17dd-4494-bffa-0a01158b6d08 None admin] RBAC: Authorizing `identity:get_user(user_id=d4f3ad3806dd4f378cf1552466225045)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:33.230229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccbaa15-17dd-4494-bffa-0a01158b6d08 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:33.232281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 82/164] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:32 2022] GET /identity/v3/users/d4f3ad3806dd4f378cf1552466225045 => generated 247 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:33.236511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4673b5-8199-44ef-b2bc-cbe5c127c081 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:33.236511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4673b5-8199-44ef-b2bc-cbe5c127c081 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:33.236511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4673b5-8199-44ef-b2bc-cbe5c127c081 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:33.240577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 83/165] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:33.245842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:33.246136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:33.246417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:33.248975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:33.249716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:33.250009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:33.250285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:33.251577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:33.251912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:33.253314 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:33.276105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 83/166] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:33.276967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:33.277920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:33.279296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:33.417974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 84/167] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:33.418927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:33.471867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcc905-87d9-4e35-92db-7ad5a20f38d8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:33.516810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:33.535324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:33.538278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:33.538278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:33.538278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:33.543983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:33.663476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:33.664983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:33.669770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c03d3-ed88-4a30-8dcb-1d163dbe9efb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:33.772465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:33.773591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:33.774016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:33.774430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:33.775860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:33.777446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:33.779517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 85/168] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:33 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:33.780570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:33.818423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 84/169] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:33.819622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:33.860819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:33.971973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75e620-67ea-4706-a4a9-0a6d0f420b46 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:33.995350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738def7-29ad-479f-bdb9-23fb8990a333 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:34.004707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:34.033028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef88e44-54d8-4af0-b4ea-7235a9fc7d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.033613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef88e44-54d8-4af0-b4ea-7235a9fc7d5e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.033900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef88e44-54d8-4af0-b4ea-7235a9fc7d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.034172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef88e44-54d8-4af0-b4ea-7235a9fc7d5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.096382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.097206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.097491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.097767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.117002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:34.130645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:34.131578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:34.132407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 86/170] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:34.133038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.176568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef88e44-54d8-4af0-b4ea-7235a9fc7d5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:34.178051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef88e44-54d8-4af0-b4ea-7235a9fc7d5e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:34.268076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:34.278546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270ae4a-23ed-4910-bda5-9f9e3d1fc960 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:34.314022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.314803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.315086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.315363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.316714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:34.318302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:34.320246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 87/171] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:34.320999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.357722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:34.386806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 85/172] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:34.388407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22144c-6e3e-4a21-ae43-d18f1ae039c0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.388705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22144c-6e3e-4a21-ae43-d18f1ae039c0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.388982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22144c-6e3e-4a21-ae43-d18f1ae039c0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.390265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 86/173] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:34.390975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22144c-6e3e-4a21-ae43-d18f1ae039c0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.446196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f4f82-be18-499e-8ce2-e3a5f782ad40 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:34.491291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.491988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.492708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.492988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.494033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:34.494342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:34.496473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3329e8-f16a-46db-bd12-648bbec75453 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:34.507408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 88/174] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:34.509006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92565b0-2a67-414c-8885-c2cf63843f94 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.509578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92565b0-2a67-414c-8885-c2cf63843f94 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.509863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92565b0-2a67-414c-8885-c2cf63843f94 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.512125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 89/175] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:34.513099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92565b0-2a67-414c-8885-c2cf63843f94 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.519100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22144c-6e3e-4a21-ae43-d18f1ae039c0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:34.557156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.557939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.558297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.558688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.559531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=d4f3ad3806dd4f378cf1552466225045) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:34.559941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:34.561482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf8c738-b7ca-42d3-9a5e-1e9b13070b1f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:34.564224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:34.568944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 87/176] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/role_assignments?user.id=d4f3ad3806dd4f378cf1552466225045&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:34.570265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5021c-90ad-4a95-8e17-82cf0e9dd595 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.570625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5021c-90ad-4a95-8e17-82cf0e9dd595 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.570972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5021c-90ad-4a95-8e17-82cf0e9dd595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.589908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-87c5021c-90ad-4a95-8e17-82cf0e9dd595 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:34.591495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-87c5021c-90ad-4a95-8e17-82cf0e9dd595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:34.638518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:34.649367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92565b0-2a67-414c-8885-c2cf63843f94 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:34.691750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.692643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.692928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.693201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] PATH_INFO: `/v3/projects/72004884fa2849369e75500d5f62a258` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.694969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] RBAC: Authorizing `identity:get_project(project_id=72004884fa2849369e75500d5f62a258)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:34.696460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:34.699141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 90/177] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/projects/72004884fa2849369e75500d5f62a258 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:34.700278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.726761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 88/178] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:34.728204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.728552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.728831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.762310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:34.763608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:34.831270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0eb0d-472a-478b-bf36-d48436a85c59 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:34.859766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 89/179] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:34.862281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.877964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:34.878996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:34.879407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:34.880030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] PATH_INFO: `/v3/users/22fb0cc1f4074e9e9e9005557751064f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:34.881131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] RBAC: Authorizing `identity:get_user(user_id=22fb0cc1f4074e9e9e9005557751064f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:34.887699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:34.887699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 91/180] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/users/22fb0cc1f4074e9e9e9005557751064f => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:34.887699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:34.931267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:34.955964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.008230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2710cd6-cf34-42fa-91a5-16d765527b0a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.011457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e0e49-876f-40f3-8d55-68ea760e37a4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.046178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6211d25-5fd0-4313-b082-332057ac5d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.047199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6211d25-5fd0-4313-b082-332057ac5d96 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.048039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6211d25-5fd0-4313-b082-332057ac5d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.048301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6211d25-5fd0-4313-b082-332057ac5d96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.048639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.049308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.049497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6211d25-5fd0-4313-b082-332057ac5d96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.049601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.049856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.050672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=72004884fa2849369e75500d5f62a258, user.id=22fb0cc1f4074e9e9e9005557751064f) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:35.050926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6211d25-5fd0-4313-b082-332057ac5d96 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.051018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.052421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.052577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 92/181] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:35.057152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daa83-0ce3-4f12-9978-d015cf739f2c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.057152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daa83-0ce3-4f12-9978-d015cf739f2c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.057445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1daa83-0ce3-4f12-9978-d015cf739f2c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.058706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 93/182] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:35.059404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1daa83-0ce3-4f12-9978-d015cf739f2c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.064555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 90/183] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:34 2022] GET /identity/v3/role_assignments?user.id=22fb0cc1f4074e9e9e9005557751064f&scope.project.id=72004884fa2849369e75500d5f62a258&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:35.065342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.184941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.192326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1daa83-0ce3-4f12-9978-d015cf739f2c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.199448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc00135-0be2-41b0-9142-02ad8e7732f1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.226362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.246379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5bb2c8-ae90-4063-9cd5-6d3b871f8673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.247046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb2c8-ae90-4063-9cd5-6d3b871f8673 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.247456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb2c8-ae90-4063-9cd5-6d3b871f8673 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.247769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5bb2c8-ae90-4063-9cd5-6d3b871f8673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.248645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.249705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.250214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.251050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.267055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:35.274762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:35.276351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 91/184] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:35.277138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.331117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb2c8-ae90-4063-9cd5-6d3b871f8673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.333731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5bb2c8-ae90-4063-9cd5-6d3b871f8673 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.388925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121182-ce7c-4347-91a3-a68551873594 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.421871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.423191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.423485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.423795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.424496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.425617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.425955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.430144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 94/185] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:35.431219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc479-3968-45b8-bc87-45da307619c8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.431510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc479-3968-45b8-bc87-45da307619c8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.431899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccc479-3968-45b8-bc87-45da307619c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.447584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e1ccc479-3968-45b8-bc87-45da307619c8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:35.448701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e1ccc479-3968-45b8-bc87-45da307619c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:35.512435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 95/186] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:35.513695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3d82d-b9f8-44cc-a880-e9233837448d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.513979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3d82d-b9f8-44cc-a880-e9233837448d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.514252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3d82d-b9f8-44cc-a880-e9233837448d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.515632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 96/187] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:35.516282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3d82d-b9f8-44cc-a880-e9233837448d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.531059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 92/188] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:35 2022] POST /identity/v3/users => generated 297 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:35.532845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.586607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.635600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3d82d-b9f8-44cc-a880-e9233837448d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.651957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1902c5c3-90fc-4420-a204-9e240600e288 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.666982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.674624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.675304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.675584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.675884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.676643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:35.676945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.678400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b83375-046f-4e9f-be99-e6a5248a6bc7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.687598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 97/189] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:35.693408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.693718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.694039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.694142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.695023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.695440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.695752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.696956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.698514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.707022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 93/190] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:35.707022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.712428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:35.713625 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:35.796895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 98/191] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:35.804319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.830883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2778f528-fb55-484d-9bc9-bbde479346f2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.841490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.876164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.876999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.877326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.877642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.888399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:35.897163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:35.898076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:35.898842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 94/192] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:35.899462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:35.934056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c68177-10f8-45a3-b016-36c34ee85ecc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:35.956599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:35.971838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:35.972712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:35.973002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:35.973247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] PATH_INFO: `/v3/users/0d7cf183a1104dc4938c87032ddc2b75` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:35.989462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] RBAC: Authorizing `identity:get_user(user_id=0d7cf183a1104dc4938c87032ddc2b75)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:35.991027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:35.993116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 99/193] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/users/0d7cf183a1104dc4938c87032ddc2b75 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:35.993703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.008120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf1029b-9139-483a-be3b-41e50665f876 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.043326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.044329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.044792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.045246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.046248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] RBAC: Adding query filter params (role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=22fb0cc1f4074e9e9e9005557751064f) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:36.046761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.048441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.054857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 95/194] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/role_assignments?user.id=22fb0cc1f4074e9e9e9005557751064f&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:36.060067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.110180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c9c14a-0ca8-4ee7-ad1f-b7c833b4f470 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.123703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:36.150770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.151496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.151808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.152087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.166028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:36.180102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:36.181035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 100/195] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:36.181814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.229788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:36.243102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab24c5-f8aa-4d35-b621-c5c5116d48dd None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.289161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee053-f915-4030-9834-009945b32a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.290582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee053-f915-4030-9834-009945b32a09 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.291077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee053-f915-4030-9834-009945b32a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.291527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fee053-f915-4030-9834-009945b32a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.292493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fee053-f915-4030-9834-009945b32a09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:36.292976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fee053-f915-4030-9834-009945b32a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.294596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fee053-f915-4030-9834-009945b32a09 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.311391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 96/196] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:36.312589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee053-f915-4030-9834-009945b32a09 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.328449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b935d-a314-4f37-a5d7-0bbc324028fc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.399471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.400445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.400958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.401411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.468025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:36.480685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fee053-f915-4030-9834-009945b32a09 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.485622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.487118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.522544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.523686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.524185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.524664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.525609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:36.526158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.527861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.528703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:36.538957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 97/197] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:36.540098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.594409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 101/198] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:36.595264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.649686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4ceec-ec72-464a-84d5-ab3c29bf4f8c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.686573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.687955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.689026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.689501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] PATH_INFO: `/v3/users/0d7cf183a1104dc4938c87032ddc2b75` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.691080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] RBAC: Authorizing `identity:get_user(user_id=0d7cf183a1104dc4938c87032ddc2b75)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.692939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.694912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 98/199] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/users/0d7cf183a1104dc4938c87032ddc2b75 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:36.696045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.702643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2944b6-621d-4aae-b4b8-3f31c6ab5905 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.725749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:36.741422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.742228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.742515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.742795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] PATH_INFO: `/v3/users/f54c2f5b0547465cb784fc695cf6e294` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.744363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] RBAC: Authorizing `identity:get_user(user_id=f54c2f5b0547465cb784fc695cf6e294)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.746449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.748299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 102/200] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/users/f54c2f5b0547465cb784fc695cf6e294 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:36.749445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.759186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:36.816216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7430e-3657-4302-a48f-2586dc1c4bed None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.861402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006abe0b-d915-42d5-ae35-34ec61392eee None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:36.862111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.862988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.863276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.863559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.866034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.867565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.869764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 99/201] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:36.870602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:36.901644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:36.902698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:36.903187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:36.903684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:36.980083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:36.981592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:36.990393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:37.005914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:37.016668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a44766-8b80-42de-b061-97cebd57b8eb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:37.059143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:37.060194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:37.060732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:37.061295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:37.117707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 103/202] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:37.117707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:37.140864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:37.143235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:37.230158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ccfa1-5bc6-4250-bdd3-5f0db6a31340 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:37.238206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:37.240874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:37.251745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 100/203] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:37.256956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:37.284706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:37.286276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:37.287012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:37.287479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:37.289261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=0d7cf183a1104dc4938c87032ddc2b75) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:37.290002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:37.291816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:37.299547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 104/204] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:37 2022] GET /identity/v3/role_assignments?user.id=0d7cf183a1104dc4938c87032ddc2b75&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:37.300738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:37.369065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842bfa25-b241-4972-b9bf-c78530eb514e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:37.406016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:37.407691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:37.408071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:37.408416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:37.410025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:37.411581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:37.413416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 101/205] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:37 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:37.418989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:37.436316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3ae3-aad1-4e85-9e1f-e72dab23b8d9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:37.481466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:37.482716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:37.483185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:37.483693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:37.484959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:37.487340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:37.489030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:37.496244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 105/206] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:37 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:37.497423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:37.535372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d589f-b6c4-422b-9a3b-a7105093f26b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:37.574427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:37.575452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:37.575958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:37.576453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:37.577483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] RBAC: Adding query filter params (role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=0d7cf183a1104dc4938c87032ddc2b75) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:37.578010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:37.580262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c2861-b532-4d35-811f-2af9c24551a2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:37.585008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:37.587200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 102/207] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:37 2022] GET /identity/v3/role_assignments?user.id=0d7cf183a1104dc4938c87032ddc2b75&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:37.664299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc1af4-2e1c-4572-847e-7438d1b8e8d7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:37.716067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:37.716732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:37.717014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:37.717289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:37.718138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] RBAC: Adding query filter params (role.id=dd5b92e530a949509eb3772b4a01aac0, scope.project.id=2a7079597609471ba368aba9a176b5f1, user.id=f54c2f5b0547465cb784fc695cf6e294) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:37.718446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:37.720119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03c347-ec39-47e2-8c80-91c5ddc23980 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:37.731107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 106/208] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:37 2022] GET /identity/v3/role_assignments?user.id=f54c2f5b0547465cb784fc695cf6e294&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=dd5b92e530a949509eb3772b4a01aac0 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:38.195298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0441-61c5-4b05-9c9b-9337c993e4e6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.195718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0441-61c5-4b05-9c9b-9337c993e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.195899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a0441-61c5-4b05-9c9b-9337c993e4e6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.197424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 103/209] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:38.203006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.203537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.204595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.225480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:38.227001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:38.277904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec60b5e-2dbb-4439-bf9d-7e3f55f3f2f8 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.279694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec60b5e-2dbb-4439-bf9d-7e3f55f3f2f8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.279694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec60b5e-2dbb-4439-bf9d-7e3f55f3f2f8 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.280087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 104/210] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:38.288974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.289292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.289602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.306653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 107/211] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:38.315680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:38.315680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:38.337844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:38.387780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:38.402674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 105/212] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:38.434357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:38.457129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d332962-707c-4aff-b53f-8d6c6fc3259d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:38.495055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:38.496817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.497400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.497873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.499344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:38.500187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:38.502891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:38.505099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 108/213] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:38.510678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:38.556432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0aa047-2acd-416b-97e4-e1b2ab3cd215 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:38.602507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:38.603578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.604090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.604626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.616256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:38.619232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:38.622571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2dd3d5-cbee-4bb2-858c-ff6fd3e90864 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:38.625518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:38.626883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:38.627643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:38.637398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 106/214] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:38.637398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:38.667043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:38.668132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.668558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.669289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.670431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:38.673077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:38.749620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c414a-0116-4132-921f-a86dc3d88fbf None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:38.762983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:38.787443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:38.788164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.788460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.788781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.789529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:38.789838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:38.791298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:38.802352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 107/215] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:38.808567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:38.813875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 109/216] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:38 2022] POST /identity/v3/users => generated 297 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:38.926288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1a62c0-2c3a-4795-b563-b5d8336f2310 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:38.963760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:38.964644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:38.964927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:38.965196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] PATH_INFO: `/v3/users/641030718f4345daae9cbd8754f1e044` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:38.981064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] RBAC: Authorizing `identity:get_user(user_id=641030718f4345daae9cbd8754f1e044)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:38.982557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:38.984519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 108/217] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:38 2022] GET /identity/v3/users/641030718f4345daae9cbd8754f1e044 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:38.990518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:39.097611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0fb57-c138-4273-b5a7-624ce43df8bf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:39.132056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:39.132758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:39.133045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:39.133326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:39.159112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:39.206837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:39.208378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:39.357037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 110/218] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:39.362997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:39.412720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:39.495395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d4623-8eeb-4774-bf68-dff1968fc13b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:39.545377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ddcc4-012a-4b95-97dc-1fcf92dd831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:39.546245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ddcc4-012a-4b95-97dc-1fcf92dd831a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:39.546530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ddcc4-012a-4b95-97dc-1fcf92dd831a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:39.546795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ddcc4-012a-4b95-97dc-1fcf92dd831a None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:39.548031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ddcc4-012a-4b95-97dc-1fcf92dd831a None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:39.549474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ddcc4-012a-4b95-97dc-1fcf92dd831a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:39.551482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 109/219] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:39 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:39.556991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:39.651241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:39.695636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613491ea-bb21-413d-a75c-b4db8f2813f6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:39.744496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:39.745181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:39.745462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:39.745730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:39.746576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=641030718f4345daae9cbd8754f1e044) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:39.746886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:39.748405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13457cc-6ca0-448e-8316-f4b2b16ed9b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:39.755615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 111/220] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:39 2022] GET /identity/v3/role_assignments?user.id=641030718f4345daae9cbd8754f1e044&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:39.965864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9850b-16db-4b87-844a-0e1b06242bf2 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:39.966204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9850b-16db-4b87-844a-0e1b06242bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:39.966379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9850b-16db-4b87-844a-0e1b06242bf2 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:39.968054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 110/221] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:39.979283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc75d26-3968-4fb9-a0ce-7a4fb3639531 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:39.979610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc75d26-3968-4fb9-a0ce-7a4fb3639531 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:39.979870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc75d26-3968-4fb9-a0ce-7a4fb3639531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:39.998791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bdc75d26-3968-4fb9-a0ce-7a4fb3639531 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:40.000016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bdc75d26-3968-4fb9-a0ce-7a4fb3639531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:40.073424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 112/222] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:40.081733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce380-7a61-4ffd-b063-9427e1771fa0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.082126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce380-7a61-4ffd-b063-9427e1771fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.082547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce380-7a61-4ffd-b063-9427e1771fa0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.084072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 111/223] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:40.089590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.089880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.090152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.101090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce380-7a61-4ffd-b063-9427e1771fa0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:40.108720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:40.110848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:40.182009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:40.224629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 113/224] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:40.272737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:40.285267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:40.288836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce380-7a61-4ffd-b063-9427e1771fa0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:40.376765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:40.377977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.378466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.378962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.405810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc127f48-e574-48b7-b1d9-b7062e230a19 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:40.425323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:40.439693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:40.440759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:40.441729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:40.444356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 112/225] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:40.450115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:40.451360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:40.452417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.453775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.456216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.477134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:40.487751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:40.488816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:40.489750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:40.498313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 114/226] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:40.505409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:40.578802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:40.628319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7a098-3a2c-477c-b820-a1f382803bef None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:40.691327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:40.693454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.695084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.695510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.696424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:40.697737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:40.700743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:40.707857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a09c-5679-4f80-bfa1-b0c9a28e1761 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:40.722257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 113/227] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:40.728799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:40.757387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:40.794453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:40.795122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.795408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.795884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.796627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:40.797079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:40.799158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:40.821809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 115/228] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:40.827415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:40.855134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db8d171-57ae-4b41-9f2e-d78a742cddcb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:40.914089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:40.915250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:40.915535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:40.916219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] PATH_INFO: `/v3/users/0d7cf183a1104dc4938c87032ddc2b75` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:40.917428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] RBAC: Authorizing `identity:get_user(user_id=0d7cf183a1104dc4938c87032ddc2b75)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:40.918887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:40.920541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 114/229] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/users/0d7cf183a1104dc4938c87032ddc2b75 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:40.926388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:41.018362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeddfc1-2f60-4fec-8d78-4ff46813aa90 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:41.023685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:41.040116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe00ba68-0018-4d4d-aaae-4debaba687d6 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:41.044081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:41.080570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:41.081761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.082228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.082730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.085096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:41.086802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:41.089034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 115/230] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:41.096189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:41.114952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:41.115855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.116196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.116473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] PATH_INFO: `/v3/users/0d7cf183a1104dc4938c87032ddc2b75` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.118306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] RBAC: Authorizing `identity:get_user(user_id=0d7cf183a1104dc4938c87032ddc2b75)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:41.119778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:41.121431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 116/231] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:40 2022] GET /identity/v3/users/0d7cf183a1104dc4938c87032ddc2b75 => generated 283 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:41.126935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:41.294317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6801a-aad6-4305-a9ac-9978b31a9ebf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:41.315022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f00fea5-31b1-4b5d-b174-b7cb7dd8c47d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:41.333352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:41.334483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.335120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.335532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.336900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:41.339044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:41.340954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 117/232] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:41 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:41.346393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:41.394042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94ef2c-f489-4efc-9c38-daf1180f31a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:41.395278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ef2c-f489-4efc-9c38-daf1180f31a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.396952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ef2c-f489-4efc-9c38-daf1180f31a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.397372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ef2c-f489-4efc-9c38-daf1180f31a8 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/0d7cf183a1104dc4938c87032ddc2b75/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.400881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94ef2c-f489-4efc-9c38-daf1180f31a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6101317b39314abbba178eaca1d5b9a7, user_id=0d7cf183a1104dc4938c87032ddc2b75, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:41.404091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94ef2c-f489-4efc-9c38-daf1180f31a8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:41.417559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:41.478320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 116/233] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:41 2022] PUT /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/0d7cf183a1104dc4938c87032ddc2b75/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:41.495991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaea333-51f5-4cc3-90e5-e166f92001b1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:41.546509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a72a19-e2a6-4da8-998c-2ae50f1ccb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:41.547734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a72a19-e2a6-4da8-998c-2ae50f1ccb04 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.548195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a72a19-e2a6-4da8-998c-2ae50f1ccb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.548608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a72a19-e2a6-4da8-998c-2ae50f1ccb04 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/0d7cf183a1104dc4938c87032ddc2b75/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.552085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a72a19-e2a6-4da8-998c-2ae50f1ccb04 None admin] RBAC: Authorizing `identity:create_grant(project_id=6101317b39314abbba178eaca1d5b9a7, user_id=0d7cf183a1104dc4938c87032ddc2b75, role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:41.553676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a72a19-e2a6-4da8-998c-2ae50f1ccb04 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:41.678329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 118/234] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:41 2022] PUT /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/0d7cf183a1104dc4938c87032ddc2b75/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:23:41.786003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bb84-6a10-4098-9fb9-ec13d3b20692 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.786377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bb84-6a10-4098-9fb9-ec13d3b20692 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.786550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2bb84-6a10-4098-9fb9-ec13d3b20692 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.788049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 117/235] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:41.793760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.794045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.794321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.815175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:41.816767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:41.847360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8a4f-f87a-4637-9500-8f77103f35f2 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.848038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8a4f-f87a-4637-9500-8f77103f35f2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.848410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e8a4f-f87a-4637-9500-8f77103f35f2 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.850259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 118/236] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:41.855898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:41.856509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:41.856993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:41.875136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:41.877066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:41.892980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:41.904847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 119/237] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:41.934691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:41.970823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:41.973455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 119/238] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:42.008843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.067104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489dbe3-31f2-4dfe-8101-1137b59ee641 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.102482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.103643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.104135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.104590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.106003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:42.107644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:42.109864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 120/239] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:42.116857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.140393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e496e-50d9-4b48-acb3-42f3b3683797 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.192914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.193607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.193892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.194163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.215597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:42.232658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:42.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:42.234355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 120/240] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:42.239054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d50ed-0e69-4895-97c4-335f1f450457 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.242566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:42.248561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.273020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:42.285425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b488a-c91a-40ac-b067-809994a00be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.286347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b488a-c91a-40ac-b067-809994a00be9 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.286769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b488a-c91a-40ac-b067-809994a00be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.287180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b488a-c91a-40ac-b067-809994a00be9 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.288186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b488a-c91a-40ac-b067-809994a00be9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:42.289963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b488a-c91a-40ac-b067-809994a00be9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:42.368666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 121/241] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:42 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:42.370434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77972-44bf-4e02-b952-5d59a8f9313b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.370866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77972-44bf-4e02-b952-5d59a8f9313b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.371283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df77972-44bf-4e02-b952-5d59a8f9313b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.372993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 122/242] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:42.379153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.379652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.380507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.421231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:42.425153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:42.431124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b37b4f-6786-41ce-9f9a-bb6e9b754adf None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.507064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.507952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.508250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.508531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.509293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:42.509603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:42.511145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:42.523492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 121/243] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:42.528992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.563960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 123/244] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:42.608371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.648168 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:42.669959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac770-7352-47f9-90a4-c6d01ed862d4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.698095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:42.705506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.706685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.707167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.707635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] PATH_INFO: `/v3/users/3096974e2e4b4b34b753c5f428b3eae6` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.724594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] RBAC: Authorizing `identity:get_user(user_id=3096974e2e4b4b34b753c5f428b3eae6)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:42.724789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f429194-4362-4016-990e-47c3a54f76e9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.726710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:42.729412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 122/245] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:42 2022] GET /identity/v3/users/3096974e2e4b4b34b753c5f428b3eae6 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:42.734956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.771223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.772204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.772558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.772911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.792973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:42.805944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:42.806815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:42.807571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 124/246] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:42.816957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:42.871608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d203-e166-4c7f-a191-3c57417d7eb5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.911155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:42.921365 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.924449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.924449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.924449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.944528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78ae75c-3f01-41b0-9020-114e22e8e25a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:42.994423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:42.995448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:42.996010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:42.996429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:42.997345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:42.997878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:42.999426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.010045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 125/247] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:43.016973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:43.018433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:43.020277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.022282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:43.138478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e28ecd-0f9d-457b-b6d4-9e4097e624bf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:43.147067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 123/248] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 20:23:43.152782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:43.179237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:43.180431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:43.180858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:43.181282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] PATH_INFO: `/v3/users/641030718f4345daae9cbd8754f1e044` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:43.182951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] RBAC: Authorizing `identity:get_user(user_id=641030718f4345daae9cbd8754f1e044)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:43.184646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.186450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:43.188750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 126/249] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:43 2022] GET /identity/v3/users/641030718f4345daae9cbd8754f1e044 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:43.194295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:43.266340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db657fe-02ae-4dc7-bc09-7d4f17ef048d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:43.303621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:43.305091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:43.305579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:43.306141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:43.307604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:43.309410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.311847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 124/250] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:43 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:43.320945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:43.366467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:43.448208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:43.454790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a324588b-f710-4acc-8542-3e19dcc348f0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:43.482415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1680e6-4177-4917-b6a3-babd49c5d9e3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:43.492718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:43.493593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:43.494459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:43.494886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:43.496087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] RBAC: Adding query filter params (role.id=dd5b92e530a949509eb3772b4a01aac0, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=3096974e2e4b4b34b753c5f428b3eae6) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:43.496548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:43.498141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.504861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 125/251] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:43 2022] GET /identity/v3/role_assignments?user.id=3096974e2e4b4b34b753c5f428b3eae6&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=dd5b92e530a949509eb3772b4a01aac0 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:43.560592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ae8a2-714c-41f1-b742-d56ce5b02852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:43.561495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ae8a2-714c-41f1-b742-d56ce5b02852 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:43.561781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ae8a2-714c-41f1-b742-d56ce5b02852 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:43.562059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ae8a2-714c-41f1-b742-d56ce5b02852 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:43.563488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ae8a2-714c-41f1-b742-d56ce5b02852 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:43.565011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ae8a2-714c-41f1-b742-d56ce5b02852 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.567072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 127/252] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:43 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:43.572822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:43.694350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1c6d9a-71f8-4648-90fe-592e9ef53701 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:43.734808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8ca2a-ba93-450a-a08b-d72fbb2dbf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:43.736531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ca2a-ba93-450a-a08b-d72fbb2dbf3b None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:43.737022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ca2a-ba93-450a-a08b-d72fbb2dbf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:43.737437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8ca2a-ba93-450a-a08b-d72fbb2dbf3b None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/641030718f4345daae9cbd8754f1e044/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:43.745059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8ca2a-ba93-450a-a08b-d72fbb2dbf3b None admin] RBAC: Authorizing `identity:create_grant(project_id=6101317b39314abbba178eaca1d5b9a7, user_id=641030718f4345daae9cbd8754f1e044, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:43.746703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8ca2a-ba93-450a-a08b-d72fbb2dbf3b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:43.843456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 126/253] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:43 2022] PUT /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/641030718f4345daae9cbd8754f1e044/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:44.484591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fcf2f-b40a-4843-b207-c3792c3f3249 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:44.485147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fcf2f-b40a-4843-b207-c3792c3f3249 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:44.485147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8fcf2f-b40a-4843-b207-c3792c3f3249 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:44.486513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 128/254] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:44.492592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:44.492894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:44.493165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:44.516758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:44.517974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:44.536913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:44.604197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 127/255] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:44.631715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fcf2f-b40a-4843-b207-c3792c3f3249 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:44.748695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8fcf2f-b40a-4843-b207-c3792c3f3249 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:44.792557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:44.793349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:44.793633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:44.793908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:44.804267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:44.808445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:44.817204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:44.824165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:44.824932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 129/256] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:44.828777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:44.957922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afda0f-be0f-4f87-86dc-1e6300f73e27 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:45.003699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:45.004302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.004587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.004864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.005610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:45.005914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:45.007300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc60c92-8a10-4be2-8257-6300ba5e5f2d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:45.016597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 128/257] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:45.022313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:45.124155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976eea4d-747e-4c83-99f7-e7ecc00bab4e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.124473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976eea4d-747e-4c83-99f7-e7ecc00bab4e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.124721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976eea4d-747e-4c83-99f7-e7ecc00bab4e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.126775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 129/258] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:45.133068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.133358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.133639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.177358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:45.178568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:45.196007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:45.245682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b15a46-96e9-44da-a9b6-1b6fb4104592 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:45.318063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:45.318868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.319153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.319428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] PATH_INFO: `/v3/users/0d7cf183a1104dc4938c87032ddc2b75` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.320958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] RBAC: Authorizing `identity:get_user(user_id=0d7cf183a1104dc4938c87032ddc2b75)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:45.322444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:45.329776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 130/259] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:45 2022] GET /identity/v3/users/0d7cf183a1104dc4938c87032ddc2b75 => generated 283 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:45.335408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:45.337048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 130/260] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:45.384459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:45.445065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:45.488857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742a20b4-51f2-4f3a-b306-b4f461453ed4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:45.493161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1685b-ee70-45a8-9dbe-bbcad8587477 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:45.538521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:45.539197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.539481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.539785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.541937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:45.542769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.543048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.543322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.569008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:45.577053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:45.577906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:45.578719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:45.588681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 131/261] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:45.589544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:45.595961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:45.658339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:45.660104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:45.764031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:45.799792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfd552-da92-4ddf-bd4b-a5faf0b1ce56 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:45.817707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 131/262] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 06 20:23:45.823321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:45.848649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:45.849342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:45.849622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:45.849895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:45.850649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:45.851067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:45.852527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:45.867220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 132/263] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:45.873005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:45.956577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:45.997921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec737c17-5da3-441f-ad92-6dc6c518e94f None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.025191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d41484-a10a-4be6-9557-46eb67d00bbc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.034333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ed756-a817-4fc5-88c6-1ede308c8ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.035469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed756-a817-4fc5-88c6-1ede308c8ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.035791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed756-a817-4fc5-88c6-1ede308c8ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.036068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ed756-a817-4fc5-88c6-1ede308c8ab4 None admin] PATH_INFO: `/v3/users/0d7cf183a1104dc4938c87032ddc2b75` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.037266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ed756-a817-4fc5-88c6-1ede308c8ab4 None admin] RBAC: Authorizing `identity:get_user(user_id=0d7cf183a1104dc4938c87032ddc2b75)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:46.038730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ed756-a817-4fc5-88c6-1ede308c8ab4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:46.040741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 133/264] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:45 2022] GET /identity/v3/users/0d7cf183a1104dc4938c87032ddc2b75 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:46.042102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f771-7c19-4931-8fa9-e909e9929940 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.042536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f771-7c19-4931-8fa9-e909e9929940 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.042955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f771-7c19-4931-8fa9-e909e9929940 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.044359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 134/265] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:46.046159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f771-7c19-4931-8fa9-e909e9929940 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.097746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0d6a59-1446-41db-8611-0160e46035c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.098509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d6a59-1446-41db-8611-0160e46035c7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.098785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d6a59-1446-41db-8611-0160e46035c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.099054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0d6a59-1446-41db-8611-0160e46035c7 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.103249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d6a59-1446-41db-8611-0160e46035c7 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:46.104718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0d6a59-1446-41db-8611-0160e46035c7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:46.106417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 132/266] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:45 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:46.110850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.111133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.111406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.149433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:46.150604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:46.198203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:46.220702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f771-7c19-4931-8fa9-e909e9929940 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.241019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 133/267] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:46.242184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.258245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:46.276981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66d98c-3818-442d-ab95-085b01152c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.277635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66d98c-3818-442d-ab95-085b01152c0e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.282428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66d98c-3818-442d-ab95-085b01152c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.282428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66d98c-3818-442d-ab95-085b01152c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.358072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a78a2c-a598-4c67-ba42-1718cb666901 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.367562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66d98c-3818-442d-ab95-085b01152c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:46.369607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66d98c-3818-442d-ab95-085b01152c0e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:46.396066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.397152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.397638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.398112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.399161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=0d7cf183a1104dc4938c87032ddc2b75) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:46.399700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:46.401355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13235750-cdd3-432c-b7d2-4f8960885aba None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:46.408973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 134/268] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/role_assignments?user.id=0d7cf183a1104dc4938c87032ddc2b75&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:46.410872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b2676-6fcc-4185-90e6-29a5b4fffa7c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.411360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b2676-6fcc-4185-90e6-29a5b4fffa7c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.411876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b2676-6fcc-4185-90e6-29a5b4fffa7c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.413685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 135/269] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:46.419047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b2676-6fcc-4185-90e6-29a5b4fffa7c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.453199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:46.498440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 135/270] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:46.503180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.503180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.503180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.532925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:46.534177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:46.572098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b2676-6fcc-4185-90e6-29a5b4fffa7c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.609689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:46.661346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.662560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.663043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.663511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.685566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:46.692301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 136/271] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:46.693179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:46.695088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:46.695956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:46.696724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:46.699256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 136/272] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:46.706092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.817521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:46.829062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39ead1-b2cb-4e04-91a6-021868fbeb8e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.834370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2988fe-0aa8-490b-a886-dc9186c82ac8 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:46.873003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.873323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:46.873788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.874071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.874346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.874803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:46.875318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:46.875556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:46.876017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:46.876949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:46.877050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:46.877550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:46.879008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 137/273] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:46.879391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:46.884028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.894266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 137/274] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:46.895506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:46.918461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.000913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf26e-915b-43af-aab2-850e30fc1612 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.012732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418bdea9-68d7-40be-b91c-53531665f2e5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.037331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.038114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.038508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.038781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.040058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.041609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:47.043565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 138/275] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:47.044101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.053142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.053790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.054071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.054343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.055195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] RBAC: Adding query filter params (role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=0d7cf183a1104dc4938c87032ddc2b75) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:47.055504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.057000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad60fc-895c-4ffd-8bd7-0675e9c172a9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:47.063309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 138/276] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:46 2022] GET /identity/v3/role_assignments?user.id=0d7cf183a1104dc4938c87032ddc2b75&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:47.064590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4e1d-1b6d-4300-9301-3d7a9c16f0e2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.064883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4e1d-1b6d-4300-9301-3d7a9c16f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.065158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a4e1d-1b6d-4300-9301-3d7a9c16f0e2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.066376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 139/277] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:47.067132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4e1d-1b6d-4300-9301-3d7a9c16f0e2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.090214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.170612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.174690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5aff4-21f5-4599-86e3-1bf8d630813e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.189669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a4e1d-1b6d-4300-9301-3d7a9c16f0e2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.216125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e0bde-0982-40d0-92b4-f026c99e2026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.217013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e0bde-0982-40d0-92b4-f026c99e2026 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.217295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e0bde-0982-40d0-92b4-f026c99e2026 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.217576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e0bde-0982-40d0-92b4-f026c99e2026 None admin] PATH_INFO: `/v3/users/911a2dc6e1564c0293e686353b68ccc5` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.230338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd728156-31c0-4430-99b2-500abf7ee75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.231101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd728156-31c0-4430-99b2-500abf7ee75a None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.231503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd728156-31c0-4430-99b2-500abf7ee75a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.231813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd728156-31c0-4430-99b2-500abf7ee75a None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.232533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd728156-31c0-4430-99b2-500abf7ee75a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.233992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd728156-31c0-4430-99b2-500abf7ee75a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:47.237709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e0bde-0982-40d0-92b4-f026c99e2026 None admin] RBAC: Authorizing `identity:get_user(user_id=911a2dc6e1564c0293e686353b68ccc5)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.239570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e0bde-0982-40d0-92b4-f026c99e2026 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:47.241563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 139/278] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/users/911a2dc6e1564c0293e686353b68ccc5 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:47.242639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095375f-01ad-4c71-ac63-00679bbc1c47 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.242925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095375f-01ad-4c71-ac63-00679bbc1c47 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.243200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095375f-01ad-4c71-ac63-00679bbc1c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.262299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3095375f-01ad-4c71-ac63-00679bbc1c47 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:47.263483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3095375f-01ad-4c71-ac63-00679bbc1c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:47.332429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 140/279] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:47.333945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29b2f-d827-4515-ae8c-18c284d2f55e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.334235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29b2f-d827-4515-ae8c-18c284d2f55e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.334514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef29b2f-d827-4515-ae8c-18c284d2f55e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.335836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 141/280] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:47.336580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef29b2f-d827-4515-ae8c-18c284d2f55e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.348194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 140/281] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:23:47 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:47.350344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.351422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.351422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.371401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:47.372791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:47.431935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.451778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef29b2f-d827-4515-ae8c-18c284d2f55e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.455577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 141/282] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:47.455958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.458397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.491584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.492290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.492597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.492889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.493952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, user.id=911a2dc6e1564c0293e686353b68ccc5, scope.system=all) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:47.494274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.495863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:47.501956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 142/283] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/role_assignments?user.id=911a2dc6e1564c0293e686353b68ccc5&scope.system=all&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:47.502806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.597602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec836dda-feb3-4ec5-8b30-d83c97e33330 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.643563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf5e87-b584-48c4-82c2-053718a8de79 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.651793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.652608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.652892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.653170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.666964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:47.675622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:47.676571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:47.677345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 142/284] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:47.684305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.694006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.694649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.694934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.695212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.711873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:47.721499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:47.722418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:47.723198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 143/285] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:47.730376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.744800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.766258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:47.828329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf546d-3ebf-4277-9442-390c34d6e02d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.889724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f5b66-d232-416e-b596-c7af930d5b83 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:47.895986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.896924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.897203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.897478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.898231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:47.898533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.904105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:47.920021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 143/286] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:47.925928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:47.970894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:47.971571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:47.971881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:47.972156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:47.972899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:47.973205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:47.974599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.000892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 144/287] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:48.002824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:48.088518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805a16f3-5673-4c3e-b2ce-4c0dd980f97a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:48.114982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:48.127686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:48.128555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:48.129136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:48.129420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] PATH_INFO: `/v3/users/3096974e2e4b4b34b753c5f428b3eae6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:48.130882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] RBAC: Authorizing `identity:get_user(user_id=3096974e2e4b4b34b753c5f428b3eae6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:48.132378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.134989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 144/288] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:47 2022] GET /identity/v3/users/3096974e2e4b4b34b753c5f428b3eae6 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:48.140809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:48.173020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:48.228451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec10638c-5f38-4306-857d-6a2af28a91a1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:48.292571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:48.293448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:48.293732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:48.294006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] PATH_INFO: `/v3/users/641030718f4345daae9cbd8754f1e044` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:48.295316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] RBAC: Authorizing `identity:get_user(user_id=641030718f4345daae9cbd8754f1e044)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:48.296855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.305857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 145/289] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:48 2022] GET /identity/v3/users/641030718f4345daae9cbd8754f1e044 => generated 297 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:48.306705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:48.334465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6faa9-e176-44a3-8451-8f5f3f8f8940 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:48.408089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:48.408963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:48.409244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:48.409518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:48.412084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:48.413531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.415401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 145/290] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:48 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:48.424039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:48.506316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18994d0b-4f7c-4993-aa6d-2bfd085d677f None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:48.539799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:48.574529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:48.575181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:48.575459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:48.575751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:48.623988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:48.654395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:48.656357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.661658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bbcf3-f5ad-4966-a103-6d6b9ac0a669 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:48.765181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 146/291] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:48.771697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:48.773085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:48.773979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:48.774261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:48.774539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/3096974e2e4b4b34b753c5f428b3eae6/roles/dd5b92e530a949509eb3772b4a01aac0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:48.780953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] RBAC: Authorizing `identity:create_grant(project_id=6101317b39314abbba178eaca1d5b9a7, user_id=3096974e2e4b4b34b753c5f428b3eae6, role_id=dd5b92e530a949509eb3772b4a01aac0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:48.782602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.824112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:48.878820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 146/292] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:23:48 2022] PUT /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7/users/3096974e2e4b4b34b753c5f428b3eae6/roles/dd5b92e530a949509eb3772b4a01aac0 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:48.900316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc7372-4924-49f2-b044-33cbc4ccae96 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:48.945187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f3cc9-b5a6-4bfb-a263-648de0ca5175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:48.946060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f3cc9-b5a6-4bfb-a263-648de0ca5175 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:48.946344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f3cc9-b5a6-4bfb-a263-648de0ca5175 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:48.946619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f3cc9-b5a6-4bfb-a263-648de0ca5175 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:48.947857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f3cc9-b5a6-4bfb-a263-648de0ca5175 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:48.949378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f3cc9-b5a6-4bfb-a263-648de0ca5175 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:48.951337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 147/293] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:48 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:48.957874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:49.066927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a8601-4ddd-447f-b3f2-0d5c075bcd6e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:49.088056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:49.109186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:49.110396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:49.110679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:49.110958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:49.111815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=641030718f4345daae9cbd8754f1e044) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:49.112130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:49.113668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19844f3a-4805-4fa5-ba01-4cd48db2bbe3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:49.124559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 147/294] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:48 2022] GET /identity/v3/role_assignments?user.id=641030718f4345daae9cbd8754f1e044&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:49.961521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e8bc-5825-4e1b-80f2-fde1353dbd23 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:49.962038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e8bc-5825-4e1b-80f2-fde1353dbd23 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:49.962038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8e8bc-5825-4e1b-80f2-fde1353dbd23 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:49.963546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 148/295] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:49.968883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e610ed-0e78-4929-9935-33c4063149cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:49.969847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e610ed-0e78-4929-9935-33c4063149cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:49.970029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e610ed-0e78-4929-9935-33c4063149cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:49.986843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-89e610ed-0e78-4929-9935-33c4063149cb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:49.988061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89e610ed-0e78-4929-9935-33c4063149cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:50.061766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 148/296] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:50.084487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e8bc-5825-4e1b-80f2-fde1353dbd23 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.091230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09ffb-f93b-42e9-83fd-4a039f28a9af None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.091230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09ffb-f93b-42e9-83fd-4a039f28a9af None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.091230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb09ffb-f93b-42e9-83fd-4a039f28a9af None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.091230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 149/297] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:23:50.098712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d9ab7-93a0-4d80-9251-69b602c21317 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.098712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d9ab7-93a0-4d80-9251-69b602c21317 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.098712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d9ab7-93a0-4d80-9251-69b602c21317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.116793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-783d9ab7-93a0-4d80-9251-69b602c21317 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:50.118768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-783d9ab7-93a0-4d80-9251-69b602c21317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:50.191922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 150/298] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:50.193295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374fff-8899-4a34-a85f-8aafd5ca6e4c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.193584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374fff-8899-4a34-a85f-8aafd5ca6e4c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.193859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c374fff-8899-4a34-a85f-8aafd5ca6e4c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.195251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 151/299] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:50.200438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.200723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.200995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.208680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8e8bc-5825-4e1b-80f2-fde1353dbd23 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.217886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:50.219532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:50.220450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:50.255994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.256779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.257060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.257332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.265033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:50.268107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:50.275922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:50.276820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:50.277659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 149/300] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:50.278292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.299193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 152/301] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:50.299927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.391559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef650c8-7478-4928-b6da-33f3f1175f75 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.411206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aea998-978b-4ec7-8037-364246490f81 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.429210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.429998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.430278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.430932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.432424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:50.434324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:50.441313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 150/302] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:50.441313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.444918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.445592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.445876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.446149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.446894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:50.447201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:50.448605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:50.457408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 153/303] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:50.458189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.492810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:50.494096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:50.557616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cbc35-c9d5-420e-826a-d947355bfd83 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.557929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36485671-acc4-4e78-9737-045e76e71d29 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.593012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.593745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.594025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.594297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] PATH_INFO: `/v3/groups` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.594552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.594966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:50.595339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.595692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.595976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.596460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:50.606589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:50.614718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:50.615594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:50.616411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 151/304] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:50.617178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.666303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 154/305] 10.222.0.41 () {60 vars in 1134 bytes} [Mon Jun 6 20:23:50 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:23:50.666898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.754059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:50.763344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0d9f0-f59b-4ebe-8a02-9ce279820683 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.791954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:50.798877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5b67e-3b05-47ad-83f0-5cbad63fe840 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.813976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.815094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.815829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.816107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] PATH_INFO: `/v3/users/911a2dc6e1564c0293e686353b68ccc5` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.817585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] RBAC: Authorizing `identity:get_user(user_id=911a2dc6e1564c0293e686353b68ccc5)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:50.820213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:50.825402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 152/306] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/users/911a2dc6e1564c0293e686353b68ccc5 => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:50.831585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.845453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:50.846482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:50.846949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:50.847500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:50.848487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:50.849055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:50.850904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:50.866498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 155/307] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:50.868017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:50.974780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a43b01-2aa9-430a-a652-bbe51dd4d799 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:50.986576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2025d0-2f5b-471e-8b6f-87cdc5508af1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:51.020385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74b1fd-3a95-475a-a19c-d8677e9c46c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:51.021472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b1fd-3a95-475a-a19c-d8677e9c46c9 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.021715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:51.022083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b1fd-3a95-475a-a19c-d8677e9c46c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.022498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74b1fd-3a95-475a-a19c-d8677e9c46c9 None admin] PATH_INFO: `/v3/system/users/911a2dc6e1564c0293e686353b68ccc5/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.022890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.023271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.023648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] PATH_INFO: `/v3/users/8a69fe8fe6ae4fcf94e838c35d105405` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.024632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b1fd-3a95-475a-a19c-d8677e9c46c9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=911a2dc6e1564c0293e686353b68ccc5, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:51.026364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74b1fd-3a95-475a-a19c-d8677e9c46c9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:51.040666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] RBAC: Authorizing `identity:get_user(user_id=8a69fe8fe6ae4fcf94e838c35d105405)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:51.048062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:51.048062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 156/308] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:50 2022] GET /identity/v3/users/8a69fe8fe6ae4fcf94e838c35d105405 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:51.050105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:51.068525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:51.129854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 153/309] 10.222.0.41 () {58 vars in 1334 bytes} [Mon Jun 6 20:23:50 2022] PUT /identity/v3/system/users/911a2dc6e1564c0293e686353b68ccc5/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:23:51.173976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2e981-d759-4a67-af15-c7e61281595a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:51.213767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:51.214747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.215272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.215770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.216832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] RBAC: Adding query filter params (role.id=dd5b92e530a949509eb3772b4a01aac0, user.id=8a69fe8fe6ae4fcf94e838c35d105405, scope.system=all) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:51.217346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:51.219598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223d201-6a9e-48c6-9c0e-c6835281f7e3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:51.226306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 157/310] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 20:23:51 2022] GET /identity/v3/role_assignments?user.id=8a69fe8fe6ae4fcf94e838c35d105405&scope.system=all&role.id=dd5b92e530a949509eb3772b4a01aac0 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:51.448543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce5059-584c-4624-be96-57e426533d6c None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.449193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce5059-584c-4624-be96-57e426533d6c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.449678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce5059-584c-4624-be96-57e426533d6c None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.451264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 154/311] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:51.457264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.457560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.457840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.477878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:51.479065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:51.549595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 158/312] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:51.572791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce5059-584c-4624-be96-57e426533d6c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:51.603679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:51.679457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce5059-584c-4624-be96-57e426533d6c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:51.714130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:51.714853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.715141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.715421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.726409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:51.734277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:51.735187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:51.735981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 155/313] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:51.741749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:51.760005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:51.843912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa7cd6-e56b-4557-8cae-2711a1103f7d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:51.879006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:51.879711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:51.880052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:51.880333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:51.881095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:51.881401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:51.882811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:51.893575 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 159/314] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:51.899093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:51.990355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:52.003029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e27fd-d2f6-4acb-9593-27e3f976b441 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:52.041213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:52.042078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:52.042357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:52.042625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] PATH_INFO: `/v3/users/3096974e2e4b4b34b753c5f428b3eae6` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:52.043838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] RBAC: Authorizing `identity:get_user(user_id=3096974e2e4b4b34b753c5f428b3eae6)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:52.045279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:52.047454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 156/315] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:51 2022] GET /identity/v3/users/3096974e2e4b4b34b753c5f428b3eae6 => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:52.052703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:52.132965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:52.155689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ffd0d-88b0-4d12-8b07-8e557e652a4f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:52.190995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:52.191651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:52.192007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:52.192285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:52.261126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:52.262538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:52.334608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:52.367931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 160/316] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:52.373734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:52.482410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a5c33-47bb-444f-9c07-afec0ae37459 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:52.519009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30fa3cd-6068-48d6-95f6-d7af33847edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:52.519812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fa3cd-6068-48d6-95f6-d7af33847edb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:52.520097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fa3cd-6068-48d6-95f6-d7af33847edb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:52.520374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30fa3cd-6068-48d6-95f6-d7af33847edb None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:52.521566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fa3cd-6068-48d6-95f6-d7af33847edb None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:52.523005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30fa3cd-6068-48d6-95f6-d7af33847edb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:52.524729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 157/317] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:52 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:52.530486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:52.639937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745bf0ae-d408-4a52-a72a-895f0d140187 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:52.677249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:52.677902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:52.678237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:52.678516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:52.679342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] RBAC: Adding query filter params (role.id=dd5b92e530a949509eb3772b4a01aac0, scope.project.id=6101317b39314abbba178eaca1d5b9a7, user.id=3096974e2e4b4b34b753c5f428b3eae6) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:52.679652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:52.681123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d935b4d-336a-46e1-be70-476d4d4e26b1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:52.687406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 161/318] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:23:52 2022] GET /identity/v3/role_assignments?user.id=3096974e2e4b4b34b753c5f428b3eae6&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=dd5b92e530a949509eb3772b4a01aac0 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:53.053697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdb518-ee1f-4fc9-a9be-466b4a98cfce None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.054211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdb518-ee1f-4fc9-a9be-466b4a98cfce None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.054316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cdb518-ee1f-4fc9-a9be-466b4a98cfce None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.055895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 158/319] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:53.061187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.061568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.061735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.078810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:53.080022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:53.108897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:53.154917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 162/320] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:53.174068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdb518-ee1f-4fc9-a9be-466b4a98cfce None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.201710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:53.284257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cdb518-ee1f-4fc9-a9be-466b4a98cfce None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:53.317932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dd634e-e5af-48ad-9396-58c6dbcc81b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:53.318640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd634e-e5af-48ad-9396-58c6dbcc81b2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.318921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd634e-e5af-48ad-9396-58c6dbcc81b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.319196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd634e-e5af-48ad-9396-58c6dbcc81b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.320387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd634e-e5af-48ad-9396-58c6dbcc81b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:53.321813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dd634e-e5af-48ad-9396-58c6dbcc81b2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:53.323563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 159/321] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:23:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:53.329163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.430412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2248ec-75ef-4a51-814d-f376a72f7f83 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:53.463486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a0e35-b7e9-4505-bb2c-ae8c3372784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:53.464177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0e35-b7e9-4505-bb2c-ae8c3372784f None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.464462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0e35-b7e9-4505-bb2c-ae8c3372784f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.464738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a0e35-b7e9-4505-bb2c-ae8c3372784f None admin] PATH_INFO: `/v3/groups` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.465432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0e35-b7e9-4505-bb2c-ae8c3372784f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:53.466860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a0e35-b7e9-4505-bb2c-ae8c3372784f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:53.473952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d832e-9194-4dd6-81d6-756d6a35ff43 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.474239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d832e-9194-4dd6-81d6-756d6a35ff43 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.474518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d832e-9194-4dd6-81d6-756d6a35ff43 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.475900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 160/322] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:23:53.480849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.481128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.481405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.498216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:53.499401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 163/323] 10.222.0.41 () {60 vars in 1134 bytes} [Mon Jun 6 20:23:53 2022] POST /identity/v3/groups => generated 226 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68ea2d-cca3-489e-801e-25f02f6e969b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68ea2d-cca3-489e-801e-25f02f6e969b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68ea2d-cca3-489e-801e-25f02f6e969b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 164/324] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.551691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.552318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.568164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:53.569311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:53.571192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 161/325] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:53.595457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.597037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:53.622712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:53.648787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 165/326] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:53.671794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.704173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e7b42-8332-4eaa-8f35-3b00ed95eab8 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:53.742743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:53.743824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.744250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.744672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.759838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:53.768729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:53.769633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:53.770406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 162/327] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:53.776331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.787112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d54d12-bd7e-428b-9854-4a6e17931cb6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:53.825046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:53.825764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.826128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.826403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.839220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:53.848677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:53.849584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:53.850352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:53.851456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 166/328] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:53.857432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.880769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:53.883162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:53.907269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7052ba8e-b994-4038-bd38-c534eb34701a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:53.954468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:53.955266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:53.955555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:53.955852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:53.956895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:53.957549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:53.959441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:53.970772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 163/329] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:53.976660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:53.989794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55e8fc-ee93-40a1-85ec-0f314b9cb17d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:54.036140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:54.036817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:54.037104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:54.037384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:54.038140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:54.038456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:54.041219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:54.053396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 167/330] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:54.062057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:54.109818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae736419-a0ac-4450-938f-3b8e1fbec8e0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:54.147551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:54.148792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:54.149355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:54.149796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] PATH_INFO: `/v3/users/911a2dc6e1564c0293e686353b68ccc5` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:54.151134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] RBAC: Authorizing `identity:get_user(user_id=911a2dc6e1564c0293e686353b68ccc5)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:54.152952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:54.155132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 164/331] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:53 2022] GET /identity/v3/users/911a2dc6e1564c0293e686353b68ccc5 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:54.164724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:54.181292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:54.194569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:54.215524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b058c6-9959-4700-98e7-f47c5a3bd358 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:54.268290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:54.269688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:54.270208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:54.270862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] PATH_INFO: `/v3/users/8a69fe8fe6ae4fcf94e838c35d105405` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:54.272703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] RBAC: Authorizing `identity:get_user(user_id=8a69fe8fe6ae4fcf94e838c35d105405)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:54.274357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:54.277589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 168/332] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:54 2022] GET /identity/v3/users/8a69fe8fe6ae4fcf94e838c35d105405 => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:54.283121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:54.343435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9701c3e2-0258-4848-9817-208ffd674be5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:54.395944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:54.396528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:54.396808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:54.397079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:54.397911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] RBAC: Adding query filter params (role.id=36b9d8febf3f4f66b9a27875e9ac1c96, user.id=911a2dc6e1564c0293e686353b68ccc5, scope.system=all) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:54.398220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:54.399677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f986344-3488-4d2c-a56a-6bedbc5159de None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:54.420359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 165/333] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 20:23:54 2022] GET /identity/v3/role_assignments?user.id=911a2dc6e1564c0293e686353b68ccc5&scope.system=all&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 510 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:54.438472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eb2feb-ad0b-4f74-8a0e-69050c47d8c1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:54.476737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdb251-269a-4f9a-8b71-1fd2b9aed00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:54.477653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb251-269a-4f9a-8b71-1fd2b9aed00c None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:54.477934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb251-269a-4f9a-8b71-1fd2b9aed00c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:54.478210 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb251-269a-4f9a-8b71-1fd2b9aed00c None admin] PATH_INFO: `/v3/system/users/8a69fe8fe6ae4fcf94e838c35d105405/roles/dd5b92e530a949509eb3772b4a01aac0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:54.479842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdb251-269a-4f9a-8b71-1fd2b9aed00c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8a69fe8fe6ae4fcf94e838c35d105405, role_id=dd5b92e530a949509eb3772b4a01aac0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:54.481271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdb251-269a-4f9a-8b71-1fd2b9aed00c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:54.569858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 169/334] 10.222.0.41 () {58 vars in 1334 bytes} [Mon Jun 6 20:23:54 2022] PUT /identity/v3/system/users/8a69fe8fe6ae4fcf94e838c35d105405/roles/dd5b92e530a949509eb3772b4a01aac0 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:23:56.431911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ec779-0cd1-456b-991f-0e1b64b131d6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:56.432475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ec779-0cd1-456b-991f-0e1b64b131d6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:56.432475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7ec779-0cd1-456b-991f-0e1b64b131d6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:56.434016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 166/335] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:56.439894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:56.440766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:56.441254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:56.458508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:56.460054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:56.463169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:56.546196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 170/336] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:56.581827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ec779-0cd1-456b-991f-0e1b64b131d6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:56.664840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:56.708417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7ec779-0cd1-456b-991f-0e1b64b131d6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:56.752388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f5912-7dfd-4c9c-a767-7fb6aba6a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:56.753212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5912-7dfd-4c9c-a767-7fb6aba6a9bc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:56.753496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5912-7dfd-4c9c-a767-7fb6aba6a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:56.753771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f5912-7dfd-4c9c-a767-7fb6aba6a9bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:56.764634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f5912-7dfd-4c9c-a767-7fb6aba6a9bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-969f5912-7dfd-4c9c-a767-7fb6aba6a9bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:56.773203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:56.774120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:56.774910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 167/337] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:56.780450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:56.856512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6d65-9a41-4bab-95df-41be07bab800 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:56.856811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6d65-9a41-4bab-95df-41be07bab800 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:56.857030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d6d65-9a41-4bab-95df-41be07bab800 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:56.858712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 168/338] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:56.878788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:56.879090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:56.879337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:56.903512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:56.904756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:56.929650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60d13a-ce5b-4aca-9494-abd217e5a08f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:57.000724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 169/339] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:23:57.004603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:57.005296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.005573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.005843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.006600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:57.006902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:57.008307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:57.023059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 171/340] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:57.028880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:57.043243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:57.046595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:57.132172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:57.198661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de72fe-133d-4281-b11d-bf1301fbaa29 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:57.265261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b8bd8-f033-4260-a990-ee7689033b64 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:57.265684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:57.266780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.267068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.267613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.349739 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:57.350599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.350881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.351151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.370563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:57.379402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:57.380327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:57.381102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:57.391224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 172/341] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:23:57.396176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:57.396531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:57.397788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:57.491076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:57.503720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:57.558552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 170/342] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:57.565761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:57.571736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f272d2-73a3-495c-9752-1fc321630c3c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:57.642345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:57.642952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.643241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.643520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.644352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:57.644665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:57.646063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4436d052-56bc-43db-b636-16dbc46acdd0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:57.659760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 173/343] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:23:57.663560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcf42d-4eed-41cb-a88d-4171d5a2803f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.665216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcf42d-4eed-41cb-a88d-4171d5a2803f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.665495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcf42d-4eed-41cb-a88d-4171d5a2803f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.666908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 174/344] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:23:57.668522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcf42d-4eed-41cb-a88d-4171d5a2803f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:57.741297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217f434-8430-4fc8-9618-91c0f3c2e57d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:57.778571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc50914-8564-41b1-aac6-51d78813dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:57.779541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc50914-8564-41b1-aac6-51d78813dd35 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.779854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc50914-8564-41b1-aac6-51d78813dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.780133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc50914-8564-41b1-aac6-51d78813dd35 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.782013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc50914-8564-41b1-aac6-51d78813dd35 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:57.783525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc50914-8564-41b1-aac6-51d78813dd35 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:57.788003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 171/345] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:57 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:57.789379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c8952-1ec9-43c0-b392-38a983f788dc None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.789669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c8952-1ec9-43c0-b392-38a983f788dc None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.789949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87c8952-1ec9-43c0-b392-38a983f788dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.806700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcf42d-4eed-41cb-a88d-4171d5a2803f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:57.808953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a87c8952-1ec9-43c0-b392-38a983f788dc None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:57.810205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a87c8952-1ec9-43c0-b392-38a983f788dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:57.838195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:57.854122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf20543-639b-4664-80b0-efa8fa5645f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:57.854832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf20543-639b-4664-80b0-efa8fa5645f7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.855117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf20543-639b-4664-80b0-efa8fa5645f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.855393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf20543-639b-4664-80b0-efa8fa5645f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.857392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:57.918366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 172/346] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:57.924296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6dc9a-e5f9-4b3c-9049-9d54ce5b7410 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:57.924583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6dc9a-e5f9-4b3c-9049-9d54ce5b7410 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:57.924857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6dc9a-e5f9-4b3c-9049-9d54ce5b7410 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:57.926429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 173/347] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:57.927144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6dc9a-e5f9-4b3c-9049-9d54ce5b7410 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:57.952085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf20543-639b-4664-80b0-efa8fa5645f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:57.954144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf20543-639b-4664-80b0-efa8fa5645f7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:58.080482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 175/348] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:58.081273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6dc9a-e5f9-4b3c-9049-9d54ce5b7410 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.088316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.088601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.088897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.111543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:58.112867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:58.126030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.127176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.127460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.127758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.133295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:58.137646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:58.139600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:58.142475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 174/349] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:23:57 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:58.143342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.178461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:58.201925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 176/350] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:58.202768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.305357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23968edd-d1db-4cd8-8393-ec9925816d25 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.344398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.344676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead29b45-9542-4ff6-aab2-141428010846 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.345693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.346155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.346590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.359977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:58.369965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:58.371067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:58.371962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 175/351] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:58.376180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.393405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.394278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.394564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.395000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.398342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:58.398342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:58.400442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 177/352] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:58.404729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.432549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:58.483635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:58.505619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296192e7-3ff6-4d9d-a797-78b1589114a7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.528362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7605cca9-433b-4173-9cd7-0acc8790ef05 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.554701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.554701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.554701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.555054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.555871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=2a7079597609471ba368aba9a176b5f1) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:58.556184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:58.557956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:58.566736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 176/353] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:58.567577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.572266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.573427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.573923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.574402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.588910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:58.598184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:58.599148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:58.599990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 178/354] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:58.601993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.719742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:58.731172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5b583-c579-4c4f-b7cb-1810d485d29d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.744673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534832-8a5d-48d4-a228-cbca04fa90bb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.778842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.779827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.780118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.780938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.781144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:58.781490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:58.782874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05507b0-286e-4ba5-9bb7-4c0c0ed3ee34 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:58.792609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 177/355] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:58.796593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.797489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.797771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.798046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.798303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f10e9-6313-4f21-8699-ff6d5bc38518 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.798617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f10e9-6313-4f21-8699-ff6d5bc38518 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.798909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f10e9-6313-4f21-8699-ff6d5bc38518 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.799391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:58.800692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 178/356] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:58.800877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:58.801858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610c118-45d8-4b06-9daa-5ffc5e66cbe4 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.802631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610c118-45d8-4b06-9daa-5ffc5e66cbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.802747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 179/357] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:58.802820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610c118-45d8-4b06-9daa-5ffc5e66cbe4 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.803771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.807856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 179/358] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:23:58.807856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610c118-45d8-4b06-9daa-5ffc5e66cbe4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:58.830111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:58.940730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb32a-b21e-4c07-8882-e98692bba6bf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.951957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610c118-45d8-4b06-9daa-5ffc5e66cbe4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:58.982952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:58.984048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:58.985093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:58.985560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:58.986320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:58.986627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:58.988098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c4ba5e-0aa4-4c3c-ae13-d58062b91b5e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:59.001303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 180/359] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:59.001303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da943806-6412-447d-9742-23d72ed927a0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.001303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da943806-6412-447d-9742-23d72ed927a0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.001303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da943806-6412-447d-9742-23d72ed927a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.006374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd56678-d64c-44a4-bfab-e6e9847e96ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.007250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56678-d64c-44a4-bfab-e6e9847e96ca None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.007648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56678-d64c-44a4-bfab-e6e9847e96ca None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.007948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd56678-d64c-44a4-bfab-e6e9847e96ca None admin] PATH_INFO: `/v3/users/8a69fe8fe6ae4fcf94e838c35d105405` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.009136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd56678-d64c-44a4-bfab-e6e9847e96ca None admin] RBAC: Authorizing `identity:get_user(user_id=8a69fe8fe6ae4fcf94e838c35d105405)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:59.011348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd56678-d64c-44a4-bfab-e6e9847e96ca None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:59.012217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 180/360] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:23:58 2022] GET /identity/v3/users/8a69fe8fe6ae4fcf94e838c35d105405 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:59.013472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.013757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.014018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.022147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-da943806-6412-447d-9742-23d72ed927a0 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:59.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da943806-6412-447d-9742-23d72ed927a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:59.033072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:23:59.034890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:23:59.036815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.099607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 181/361] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:59.100860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da943806-6412-447d-9742-23d72ed927a0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.118837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 181/362] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:23:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:23:59.119819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.135706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.238548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da943806-6412-447d-9742-23d72ed927a0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:59.258229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf9cc6f-8083-4994-8f03-577b958da495 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:59.287424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.288125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.288448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.288732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.289557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=2a7079597609471ba368aba9a176b5f1) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:59.289867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:59.291296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:59.302011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 182/363] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:59.302011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.312027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.315779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.316446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.316737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.317013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.399966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:59.401450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:59.410978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.429471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27835679-72df-4a0f-8e2f-58c58ef6933b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:59.478491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.479181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.479466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.479841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.480698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] RBAC: Adding query filter params (role.id=dd5b92e530a949509eb3772b4a01aac0, user.id=8a69fe8fe6ae4fcf94e838c35d105405, scope.system=all) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:23:59.481322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:59.483655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:59.501045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 183/364] 10.222.0.41 () {56 vars in 1308 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/role_assignments?user.id=8a69fe8fe6ae4fcf94e838c35d105405&scope.system=all&role.id=dd5b92e530a949509eb3772b4a01aac0 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:59.504670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.551139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 182/365] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:23:59.552042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.622033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.691527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c415d-c349-4739-94a7-4d918af80169 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:59.742700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.743886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.744685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.745166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.763640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:59.773396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:59.774558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:23:59.775550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:23:59.778597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 184/366] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:59.780036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.786627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a8e4b-631a-40c3-af01-e92ecb9cce45 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:59.802604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.845647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.846457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.846738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.847012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.864660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:23:59.876664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:23:59.877594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:23:59.878387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:23:59.883559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 183/367] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:23:59.883559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:23:59.918384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e92e4f-47d6-4622-aa16-a122270e6188 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:23:59.952073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:23:59.963688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:23:59.965072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:23:59.965371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:23:59.965646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:23:59.967111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:23:59.969847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:23:59.972426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 185/368] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:23:59.973787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.029437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25af48e-3172-4366-af20-d72736e0a087 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.075432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.076170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.076457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.076735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.077485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:00.077790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.079282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.088709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 184/369] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:00.089349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.100768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f7b1e-3ec0-4296-9194-0eb0182075aa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.130214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.140545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.141213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.141546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.141861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.142573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:00.142878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.144356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.155054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 186/370] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:23:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:00.156695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.226810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457971a-baed-4568-9218-d09290cd5aff None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.267894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.268759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.269042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.269318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.270491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.271970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.273026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.273968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 185/371] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:00.276975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.299078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9498e33-f64e-4cab-9d98-9fd29effc935 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.350744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.351402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.351722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.351988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.404786 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.412053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4335d27-0882-4887-bc51-7aa989f6ad26 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.433928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.435392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.449497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.450539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.451065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.451535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.495630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.527913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.529619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.544552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 187/372] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:00.548606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.633185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.640878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 186/373] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:00.641787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.657596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b39150-6fbc-4acc-8cd7-fb494a3b102f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.700034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.701065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.701596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.702082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.703197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=6101317b39314abbba178eaca1d5b9a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:00.703778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.705493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.713649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 188/374] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:00.714946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.748861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.769233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5cc685-6bbb-44a1-973d-4758a7a95ce5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.825445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.826551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.827123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.828503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] PATH_INFO: `/v3/projects/284b2c60254e43d98184156216a03e15` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.830235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] RBAC: Authorizing `identity:get_project(project_id=284b2c60254e43d98184156216a03e15)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.832007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.833987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 187/375] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/projects/284b2c60254e43d98184156216a03e15 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:00.843445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:00.905814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08655bbd-417f-461d-a5c6-9f4881bcfcff None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:00.985446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:00.985924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:00.986755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:00.986875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:00.987103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:00.988553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:00.990005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:00.991776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 189/376] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.003984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:01.030774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651c102-9b1f-4615-b3ae-a23498f9c99b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.074990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.075960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.076241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.076513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] PATH_INFO: `/v3/groups/c928f20b08114c09b233bf0d1228204b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.085948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] RBAC: Authorizing `identity:get_group(group_id=c928f20b08114c09b233bf0d1228204b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:01.087405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:01.090778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 188/377] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/groups/c928f20b08114c09b233bf0d1228204b => generated 229 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.104256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:01.106634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:01.120825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a23de-35c1-4344-be50-6ce2bf5536a8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.162401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9f5cd-6076-4efe-8620-b77718d1c139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.163135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f5cd-6076-4efe-8620-b77718d1c139 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.163418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f5cd-6076-4efe-8620-b77718d1c139 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.163954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9f5cd-6076-4efe-8620-b77718d1c139 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.165762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9f5cd-6076-4efe-8620-b77718d1c139 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:01.167218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9f5cd-6076-4efe-8620-b77718d1c139 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:01.170606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 190/378] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:00 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.170606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3fc52-17b6-4f90-95ce-cf1e3c053d37 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.170606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3fc52-17b6-4f90-95ce-cf1e3c053d37 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.170862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f3fc52-17b6-4f90-95ce-cf1e3c053d37 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.174885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 191/379] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:01.176321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3fc52-17b6-4f90-95ce-cf1e3c053d37 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:01.238677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ba9b3-c916-43c5-b4f8-cf66c8664716 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.270438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:01.281763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.282838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.294923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.294923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.294923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] RBAC: Adding query filter params (group.id=c928f20b08114c09b233bf0d1228204b, role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=284b2c60254e43d98184156216a03e15) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:01.294923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:01.294923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae9125-b795-4204-95f5-1c42a926e16e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:01.295279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f3fc52-17b6-4f90-95ce-cf1e3c053d37 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.295352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 189/380] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:01 2022] GET /identity/v3/role_assignments?group.id=c928f20b08114c09b233bf0d1228204b&scope.project.id=284b2c60254e43d98184156216a03e15&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.295352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.295956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.296525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.317514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:01.319088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:01.358032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.358703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.358991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.359282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.360149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=6101317b39314abbba178eaca1d5b9a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:01.360460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:01.361892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:01.372542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 192/381] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:01 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.403364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 190/382] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:01.430706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:01.452330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:01.583171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98b6ea-f937-48e0-a618-ecf6c22aff6b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.631250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.632101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.632386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.632662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.644325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:01.652504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:01.653391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:01.654179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 193/383] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:01.660271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:01.778374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaa7dfe-d7a5-45fd-9f68-3566a503830f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.803719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:01.817240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.818174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.818635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.819113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.820059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:01.830442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:01.830442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:01.834764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 191/384] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.840705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:01.928186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:01.949642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953137e-a030-4653-879a-15f0b0d57ecb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:01.986732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641d7e2-e74f-4e06-a420-378e23838e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:01.987564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641d7e2-e74f-4e06-a420-378e23838e93 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:01.987909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641d7e2-e74f-4e06-a420-378e23838e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:01.988187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641d7e2-e74f-4e06-a420-378e23838e93 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:01.989364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641d7e2-e74f-4e06-a420-378e23838e93 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:01.990802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641d7e2-e74f-4e06-a420-378e23838e93 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:01.992776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 194/385] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:01 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:01.998186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:02.169658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e519d-ce2d-48c7-a7a8-982569457612 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:02.215538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a51b0f-77ac-4fd7-9a3f-6d18e634522f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:02.215863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a51b0f-77ac-4fd7-9a3f-6d18e634522f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:02.216125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a51b0f-77ac-4fd7-9a3f-6d18e634522f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:02.216565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:02.217640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:02.217743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 195/386] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:24:02.217814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:02.217990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:02.221707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:02.222980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:02.223173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:02.223328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:02.223618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:02.225638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 192/387] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:01 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:02.231074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:02.257714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:02.258916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:02.348008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:02.410482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 196/388] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:02.437845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:02.469900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf7ed57-8a02-47b4-a78a-8051abe7bbc5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:02.542265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:02.543509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:02.544182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:02.544662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:02.552462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7079597609471ba368aba9a176b5f1, group_id=3d6bb9493b404efdb1da4f963b6a3108, role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:02.553920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:02.615366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:02.642340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e8e308-1c07-46f6-b9a0-46cfddb19596 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:02.649063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 193/389] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 20:24:02 2022] PUT /identity/v3/projects/2a7079597609471ba368aba9a176b5f1/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:24:02.691561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:02.692334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:02.695158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:02.695484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:02.716502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:02.730595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:02.731551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:02.732353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 197/390] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:02.739679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:02.882647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae24646-1a2d-4936-b843-71097f013a1e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:02.902330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:02.935497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:02.936272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:02.936554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:02.936937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:02.937753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:02.938056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:02.939427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:02.955116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 194/391] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:24:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:02.964776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:03.098244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b38197-9c3c-4d61-ab46-4bcd19b9af13 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:03.133518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:03.134733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:03.135237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:03.135769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:03.137177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:03.138821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:03.141658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 198/392] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:02 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:03.147423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:03.290137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72a38a-7b70-4630-b648-3a6c689696c4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:03.339091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af79384-ba36-48ff-ba4d-80094a2f26bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:03.339996 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af79384-ba36-48ff-ba4d-80094a2f26bc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:03.340283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af79384-ba36-48ff-ba4d-80094a2f26bc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:03.340558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af79384-ba36-48ff-ba4d-80094a2f26bc None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:03.341766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af79384-ba36-48ff-ba4d-80094a2f26bc None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:03.343205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af79384-ba36-48ff-ba4d-80094a2f26bc None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:03.345274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 195/393] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:03 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:03.351177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:03.359798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:03.543577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52125b87-0db5-49b1-9d31-bc9c9a17155b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:03.613259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209d733-f5f1-48c7-80c3-1e2127d4801e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:03.614524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209d733-f5f1-48c7-80c3-1e2127d4801e None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:03.615014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209d733-f5f1-48c7-80c3-1e2127d4801e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:03.615514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209d733-f5f1-48c7-80c3-1e2127d4801e None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:03.617811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209d733-f5f1-48c7-80c3-1e2127d4801e None admin] RBAC: Authorizing `identity:create_grant(project_id=2a7079597609471ba368aba9a176b5f1, group_id=3d6bb9493b404efdb1da4f963b6a3108, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:03.619436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209d733-f5f1-48c7-80c3-1e2127d4801e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:03.699005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 199/394] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 20:24:03 2022] PUT /identity/v3/projects/2a7079597609471ba368aba9a176b5f1/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:03.892948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6352e-8b42-451e-8e3d-3a36943c4f46 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:03.893465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6352e-8b42-451e-8e3d-3a36943c4f46 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:03.893465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6352e-8b42-451e-8e3d-3a36943c4f46 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:03.895120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 196/395] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:03.900358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:03.900936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:03.901428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:03.924894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:03.926439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:03.998338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:04.006757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 200/396] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:04.030631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6352e-8b42-451e-8e3d-3a36943c4f46 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.092121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:04.172994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6352e-8b42-451e-8e3d-3a36943c4f46 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.215310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7fc10e-0bff-456f-b0a2-de53d7d491f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:04.216159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fc10e-0bff-456f-b0a2-de53d7d491f2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.216447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fc10e-0bff-456f-b0a2-de53d7d491f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.216719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7fc10e-0bff-456f-b0a2-de53d7d491f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.241667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7fc10e-0bff-456f-b0a2-de53d7d491f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-0d7fc10e-0bff-456f-b0a2-de53d7d491f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:04.249877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:04.250732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:04.251495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:04.254055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 197/397] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:04.259423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.409761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173da5d-229b-415b-ba27-aed8fea4076e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.410277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173da5d-229b-415b-ba27-aed8fea4076e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.410808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173da5d-229b-415b-ba27-aed8fea4076e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.412784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f993-b417-44c7-9f1e-254e9a8ec2ec None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.416841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 198/398] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:04.437316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5c963-8fc3-4058-9cb8-ca015ffe1a29 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.437546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5c963-8fc3-4058-9cb8-ca015ffe1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.437810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5c963-8fc3-4058-9cb8-ca015ffe1a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.454322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:04.454998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.455276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.455548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.456316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:04.456617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:04.458137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c59b80a-bfc3-4198-ba85-9988a8e526f5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:04.458891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4ab5c963-8fc3-4058-9cb8-ca015ffe1a29 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:04.460684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4ab5c963-8fc3-4058-9cb8-ca015ffe1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:04.470356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 201/399] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:24:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:04.471457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b315-dd54-412c-9c32-2a90d4771f6a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.471835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b315-dd54-412c-9c32-2a90d4771f6a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.472115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b315-dd54-412c-9c32-2a90d4771f6a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.473342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 202/400] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:04.477542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b315-dd54-412c-9c32-2a90d4771f6a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.536028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:04.543781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 199/401] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:04.545269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.545873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.546457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.564183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:04.566598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:04.602530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b315-dd54-412c-9c32-2a90d4771f6a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.624023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:04.640251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:04.642143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.642621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.643111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.644505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:04.646144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:04.646591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 200/402] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:04.648530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 203/403] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:04 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:04.648652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.660666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.769957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c041ffd-a81f-4026-965f-5772ef6b0ac8 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.785259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481ad62-0802-4104-8799-38ae12a71d75 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.806835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:04.807916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.808387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.808837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.810502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:04.825153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:04.831141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:04.832038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:04.832319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:04.832589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:04.835675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:04.837096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:04.838767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 204/404] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:04 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:04.839839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:04.840710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:04.841825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 201/405] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:04.842984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.844922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:04.880171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:04.960634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae468a01-876d-4f51-81e7-8b6bd0af6bdc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.980763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9038d96d-719a-4269-aa20-5e2d18296c51 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:04.998521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:04.999717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.000774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.001228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.003390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] RBAC: Authorizing `identity:create_grant(project_id=6101317b39314abbba178eaca1d5b9a7, group_id=3d6bb9493b404efdb1da4f963b6a3108, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.005114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.025977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.025977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.025977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.025977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.035220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:05.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:05.044494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:05.045664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 205/406] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:05.046317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.093106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 202/407] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 20:24:04 2022] PUT /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:05.094129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.119254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:05.200380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:05.215629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bddc9cf-5cc5-4768-b6b8-834375a5cb1a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.222496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066448f3-b826-4665-85ee-40bd705a4c6b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.271605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.272884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.273371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.273845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.274824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:05.275346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.277190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.293832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 203/408] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:05.295649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.349009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.349672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.349958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.350242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.351013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:05.351325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.352800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.375039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 206/409] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:05.380289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.416240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1278cc-b2fd-4b6e-b931-599cab45a8fb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.469914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9ed10-d839-4070-8109-91f4b5da8497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.470862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ed10-d839-4070-8109-91f4b5da8497 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.471152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ed10-d839-4070-8109-91f4b5da8497 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.471425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9ed10-d839-4070-8109-91f4b5da8497 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.472748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9ed10-d839-4070-8109-91f4b5da8497 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.474549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9ed10-d839-4070-8109-91f4b5da8497 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.481554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 204/410] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:05.481554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7d271-4e37-4739-8c2a-b8d245d5ecac None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.481554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7d271-4e37-4739-8c2a-b8d245d5ecac None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.481554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7d271-4e37-4739-8c2a-b8d245d5ecac None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.481554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 205/411] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:05.485284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7d271-4e37-4739-8c2a-b8d245d5ecac None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.502563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:05.518747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef1cc6-8714-4018-b856-26178fcd6f90 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.535403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:05.557058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313894dd-4eb4-4b9d-9e53-8e69d0064c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.558240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313894dd-4eb4-4b9d-9e53-8e69d0064c80 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.558720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313894dd-4eb4-4b9d-9e53-8e69d0064c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.559189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313894dd-4eb4-4b9d-9e53-8e69d0064c80 None admin] PATH_INFO: `/v3/groups/c928f20b08114c09b233bf0d1228204b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.560618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313894dd-4eb4-4b9d-9e53-8e69d0064c80 None admin] RBAC: Authorizing `identity:get_group(group_id=c928f20b08114c09b233bf0d1228204b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.562373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313894dd-4eb4-4b9d-9e53-8e69d0064c80 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.564609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 207/412] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/groups/c928f20b08114c09b233bf0d1228204b => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:05.565977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.566476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.566944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.584845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:05.586459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:05.599356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7d271-4e37-4739-8c2a-b8d245d5ecac None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.634941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.635771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.636127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.636433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.637611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.639165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.646139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 206/413] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:05.646139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.666625 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 208/414] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:05.667825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.761163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:05.771283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d002029-81e7-42a8-ad2d-6551d12381df None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.786028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:05.789077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96654a56-2cae-4344-adc3-2f645d7c3be0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:05.816485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.817471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.817754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.818035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.821138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6101317b39314abbba178eaca1d5b9a7, group_id=3d6bb9493b404efdb1da4f963b6a3108, role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.823298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.834131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:05.835054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:05.835419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:05.835802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] PATH_INFO: `/v3/projects/284b2c60254e43d98184156216a03e15` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:05.837026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] RBAC: Authorizing `identity:get_project(project_id=284b2c60254e43d98184156216a03e15)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:05.838523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:05.840536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 207/415] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/projects/284b2c60254e43d98184156216a03e15 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:05.841218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:05.885587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 209/416] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 20:24:05 2022] PUT /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7/groups/3d6bb9493b404efdb1da4f963b6a3108/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:05.886844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:06.006280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba07e54-3c16-4a18-a7cf-8c44bb6f769a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:06.020238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb0878-6e5f-4670-b8d3-ee3b7726cff7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:06.065166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:06.066960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.067463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.068163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] PATH_INFO: `/v3/projects/284b2c60254e43d98184156216a03e15/groups/c928f20b08114c09b233bf0d1228204b/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.071356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] RBAC: Authorizing `identity:create_grant(project_id=284b2c60254e43d98184156216a03e15, group_id=c928f20b08114c09b233bf0d1228204b, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:06.073384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:06.086359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:06.087539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.088039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.088538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.102386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:06.112202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:06.113155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:06.113993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:06.116914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 208/417] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:06.122491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:06.135595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:06.151640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 210/418] 10.222.0.41 () {58 vars in 1442 bytes} [Mon Jun 6 20:24:05 2022] PUT /identity/v3/projects/284b2c60254e43d98184156216a03e15/groups/c928f20b08114c09b233bf0d1228204b/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:06.256180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4327d7b5-4dce-4fb0-b04a-d32653221b38 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:06.294524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:06.295210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.295597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.295910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.296711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:06.297029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:06.298479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b56d56-2885-4c6f-ae49-83f63d50952c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:06.308103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 209/419] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:06.314191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:06.360024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:06.465069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddae2f-5806-48fa-a372-cc4f0402d1aa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:06.507449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a596f-3782-467d-b13e-7bdef3683532 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.507791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a596f-3782-467d-b13e-7bdef3683532 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.508041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a596f-3782-467d-b13e-7bdef3683532 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.509544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 210/420] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:06.514809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.515377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.515968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.526566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006005c6-402a-466c-a88c-040904907027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:06.527219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006005c6-402a-466c-a88c-040904907027 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.527500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006005c6-402a-466c-a88c-040904907027 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.527813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006005c6-402a-466c-a88c-040904907027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.559192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:06.560430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:06.641554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006005c6-402a-466c-a88c-040904907027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:06.642935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006005c6-402a-466c-a88c-040904907027 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:06.678806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:06.685691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 211/421] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:06.708563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:06.730762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:06.787611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 211/422] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:06.796357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006005c6-402a-466c-a88c-040904907027 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:06.844226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2140e0-e2d3-461d-87a5-72b82cbb3d77 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:06.879630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:06.880447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:06.880729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:06.881003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:06.891542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:06.899442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:06.900320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:06.901095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 212/423] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:06.906970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:07.010815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006005c6-402a-466c-a88c-040904907027 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.025161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:07.047378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:07.053560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.054470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.054756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.055025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.056400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.058054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.060083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 212/424] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:06 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:07.066191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:07.120956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b65a1c9-55f0-4fd5-8a00-b37d5cf96ecd None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.181373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.181957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.182238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.182512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.183245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:07.183552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.185094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.205216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 213/425] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:24:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:07.210633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:07.229262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ac4d9-4f71-41e9-ba1f-fa3f20d94947 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.320931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.321811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.322095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.322376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.328915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.330380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.332402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 213/426] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:07 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:07.339873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:07.350968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0425bf-4644-44b7-b607-ae6859f49d86 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.374246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:07.386119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.386932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.387028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.387306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.464292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.465767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.535720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:07.569684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 214/427] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:07.584722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:07.596175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9814-d3d5-4dfe-b8ec-052198450044 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.628025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:07.632477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.633109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.633394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.633670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.634503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=2a7079597609471ba368aba9a176b5f1) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:07.634820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.641545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.655135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 214/428] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:07 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 611 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:07.755800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55a7fda-44ba-4dd8-8475-2ad57b81ecfa None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.802086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0dfcdb-43c8-4cbd-9c61-6b1ea9750290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.802967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dfcdb-43c8-4cbd-9c61-6b1ea9750290 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.803252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dfcdb-43c8-4cbd-9c61-6b1ea9750290 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.803529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0dfcdb-43c8-4cbd-9c61-6b1ea9750290 None admin] PATH_INFO: `/v3/projects/2a7079597609471ba368aba9a176b5f1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.804807 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0dfcdb-43c8-4cbd-9c61-6b1ea9750290 None admin] RBAC: Authorizing `identity:get_project(project_id=2a7079597609471ba368aba9a176b5f1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.806275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0dfcdb-43c8-4cbd-9c61-6b1ea9750290 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.808064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 215/429] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:07 2022] GET /identity/v3/projects/2a7079597609471ba368aba9a176b5f1 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:07.814260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:07.884758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d96d9-c841-4812-9c93-0ea385991a3f None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.885070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d96d9-c841-4812-9c93-0ea385991a3f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.885343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d96d9-c841-4812-9c93-0ea385991a3f None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.886962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 216/430] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:24:07.891931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.892216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.892492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.911553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:07.912962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:07.944956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f71f6-c17e-43c4-8f87-d62525442943 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:07.982677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:07.989530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:07.990483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:07.990831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:07.991107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:07.992646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:07.994245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:07.996120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 215/431] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:07 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:08.012407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 217/432] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:08.012546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.026366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.057026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:08.142465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df56935-5400-4429-9607-343e4d14b633 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:08.151361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c1a2d-82a1-4a1d-b67c-f16ab6811ee1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:08.186069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:08.187031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.187908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.188194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.189241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=2a7079597609471ba368aba9a176b5f1) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:08.189559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:08.191390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:08.192621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:08.193622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.194097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.194590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.198847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 216/433] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=2a7079597609471ba368aba9a176b5f1&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:08.219569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:08.233537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:08.235117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 06 20:24:08.236324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:08.243827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 218/434] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:08.250751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.320868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:08.365869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47e4d1-42c3-44ce-b063-7e0083ff44cd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:08.410808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:08.411492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.411806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.412086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.412847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:08.413155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:08.414533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9267053-278c-4329-adcd-69f9b9dd99f3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:08.424260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 217/435] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:08.430013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.523683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:08.543253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91405349-dac3-444a-9c18-1ada065f39bc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:08.578721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409521e5-87a9-4590-898b-b59e7b2a5e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:08.579297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409521e5-87a9-4590-898b-b59e7b2a5e1a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.579586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409521e5-87a9-4590-898b-b59e7b2a5e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.579888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409521e5-87a9-4590-898b-b59e7b2a5e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.650065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409521e5-87a9-4590-898b-b59e7b2a5e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:08.651540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409521e5-87a9-4590-898b-b59e7b2a5e1a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:08.698572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f93418-58cc-4387-a0f5-ee45097d6f86 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.698877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f93418-58cc-4387-a0f5-ee45097d6f86 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.699142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f93418-58cc-4387-a0f5-ee45097d6f86 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.700681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 218/436] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:08.705847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406172-d81f-45eb-9768-b8a533ed2856 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.706134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406172-d81f-45eb-9768-b8a533ed2856 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.706417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7406172-d81f-45eb-9768-b8a533ed2856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.724817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a7406172-d81f-45eb-9768-b8a533ed2856 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:08.726774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a7406172-d81f-45eb-9768-b8a533ed2856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:08.737922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:08.759165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 219/437] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:08.761222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539ae0b-9b6e-4981-ac88-1e817a8922da None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.761775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539ae0b-9b6e-4981-ac88-1e817a8922da None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.761775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539ae0b-9b6e-4981-ac88-1e817a8922da None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.762982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 220/438] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:08.767274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539ae0b-9b6e-4981-ac88-1e817a8922da None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.813254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 219/439] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:08.815033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.815525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.816536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.838064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:08.839518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:08.895433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539ae0b-9b6e-4981-ac88-1e817a8922da None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:08.901040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:08.918468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 220/440] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:08.919285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.932239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:08.933147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:08.933428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:08.933701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:08.934914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:08.936369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:08.940174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 221/441] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:08.941950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:08.990452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.016531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2303ca50-4254-4ea0-826c-769d307cd9a5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.039244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af777818-8f0e-4eec-81cb-a23da02cb569 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.049986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.050782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.051062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.051331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.061034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:09.069705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:09.070597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:09.071399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 221/442] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:09.072087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.077344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.078754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.079035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.079307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.090788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:09.098364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:09.099334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:09.100157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 222/443] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:09.100788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.121879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.174507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78d97d-b03d-4690-a988-79c93ec29d42 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.196558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb24cdd-00a1-44ab-8c7c-dd9f1664bfda None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.206069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.207868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.208720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.209001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.209275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.210444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.211917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.213513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 222/444] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:09.214258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.230624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.231280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.231559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.231867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.232594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:09.232896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.235780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.243360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 223/445] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:09.244219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.321171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36937cc5-3492-43ee-82bf-d12f7b6dd4b0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.334970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.342292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291aff15-b411-4e32-9ddd-e4ffe27708d5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.353973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.354626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.354905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.355178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.355961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:09.356262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.357663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.366433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 223/446] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:09.367247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.380079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.380721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.381002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.381273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.382090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=36b9d8febf3f4f66b9a27875e9ac1c96, scope.project.id=6101317b39314abbba178eaca1d5b9a7) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:09.382397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.383845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.390654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 224/447] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=36b9d8febf3f4f66b9a27875e9ac1c96 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:09.390654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.427318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.479281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c66c8d-41c1-48ec-9c1d-4cf6f8f96819 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.499437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893e068-056f-453a-8715-c512c3a2ba62 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.519455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.520474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.520939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.521360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.534685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.535357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.535644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.535971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.574731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.600305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.602663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.614648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.616152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.657519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.716363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 224/448] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:09.722446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.739500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 225/449] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:09.745148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.811703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.832762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e36c0e-6cfc-453e-a247-a27380fdcc2c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.846321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14760b6a-c01f-4101-83e3-6147491c6fb1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.866670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.867462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.867771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.868048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] PATH_INFO: `/v3/projects/6101317b39314abbba178eaca1d5b9a7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.869226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] RBAC: Authorizing `identity:get_project(project_id=6101317b39314abbba178eaca1d5b9a7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.870651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.872355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 225/450] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/projects/6101317b39314abbba178eaca1d5b9a7 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:09.877885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.879514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:09.880383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:09.880663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:09.880933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] PATH_INFO: `/v3/projects/284b2c60254e43d98184156216a03e15` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:09.882089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] RBAC: Authorizing `identity:get_project(project_id=284b2c60254e43d98184156216a03e15)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:09.883501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:09.885186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 226/451] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/projects/284b2c60254e43d98184156216a03e15 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:09.890270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:09.892935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:09.974266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9e842-389c-4c66-a116-24632fe28b09 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:09.988596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b945af7-8b45-4ab3-aeae-55e6706843b2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:10.007582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:10.008449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:10.008732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:10.009008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] PATH_INFO: `/v3/groups/3d6bb9493b404efdb1da4f963b6a3108` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:10.010171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] RBAC: Authorizing `identity:get_group(group_id=3d6bb9493b404efdb1da4f963b6a3108)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:10.011604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:10.013218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 226/452] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/groups/3d6bb9493b404efdb1da4f963b6a3108 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:10.018797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:10.020405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:10.021245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:10.021528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:10.021802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] PATH_INFO: `/v3/groups/c928f20b08114c09b233bf0d1228204b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:10.022962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] RBAC: Authorizing `identity:get_group(group_id=c928f20b08114c09b233bf0d1228204b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:10.024408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:10.026029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 227/453] 10.222.0.41 () {56 vars in 1180 bytes} [Mon Jun 6 20:24:09 2022] GET /identity/v3/groups/c928f20b08114c09b233bf0d1228204b => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:10.032035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:10.032904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:10.099419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:10.117471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f18b22-cea4-44a1-a1ce-fb5218331493 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:10.129610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3166a92e-2a6e-40bb-b0fe-524f0160bd7d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:10.150368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:10.151019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:10.151303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:10.151623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:10.152452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] RBAC: Adding query filter params (group.id=3d6bb9493b404efdb1da4f963b6a3108, role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=6101317b39314abbba178eaca1d5b9a7) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:10.152760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:10.154183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5cc137-9c4a-4d75-ac6e-1df93f3964a7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:10.160900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 227/454] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:10 2022] GET /identity/v3/role_assignments?group.id=3d6bb9493b404efdb1da4f963b6a3108&scope.project.id=6101317b39314abbba178eaca1d5b9a7&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:10.166682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:10.167352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:10.167638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:10.167947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:10.168791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] RBAC: Adding query filter params (group.id=c928f20b08114c09b233bf0d1228204b, role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=284b2c60254e43d98184156216a03e15) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:10.169103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:10.170561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36884d71-142b-48c7-b74c-3b6a1fb4721d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:10.187701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 228/455] 10.222.0.41 () {56 vars in 1376 bytes} [Mon Jun 6 20:24:10 2022] GET /identity/v3/role_assignments?group.id=c928f20b08114c09b233bf0d1228204b&scope.project.id=284b2c60254e43d98184156216a03e15&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:12.954674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d1301-b4b3-4674-8b4a-88301b3f67c4 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:12.955640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d1301-b4b3-4674-8b4a-88301b3f67c4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:12.956252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d1301-b4b3-4674-8b4a-88301b3f67c4 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:12.958254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 228/456] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:12.963654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:12.964545 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:12.964997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:12.992738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:12.994427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:13.086215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 229/457] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:13.123234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d1301-b4b3-4674-8b4a-88301b3f67c4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:13.252069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:13.282045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d1301-b4b3-4674-8b4a-88301b3f67c4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:13.325414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:13.326505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.326958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.327396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.338709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:13.340413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1ecd0f1a-a5c7-48f9-9144-bfd17386fa84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:13.358560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:13.359463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:13.360326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:13.360326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:13.360326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:13.360326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:13.362240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 229/458] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:13.367597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:13.379212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfec31-6ea2-4b97-b397-1d48612efaae None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.379493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfec31-6ea2-4b97-b397-1d48612efaae None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.379799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfec31-6ea2-4b97-b397-1d48612efaae None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.381549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 230/459] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:13.387959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00462-1585-4a34-b814-f6be8b00b8e8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.388239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00462-1585-4a34-b814-f6be8b00b8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.388510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a00462-1585-4a34-b814-f6be8b00b8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.418173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-80a00462-1585-4a34-b814-f6be8b00b8e8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:13.419360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-80a00462-1585-4a34-b814-f6be8b00b8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:13.482546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:13.519060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 231/460] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:13.520629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890ae22-ed3e-4fc6-a8ef-6bfcd556dd54 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.520923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890ae22-ed3e-4fc6-a8ef-6bfcd556dd54 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.521559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f890ae22-ed3e-4fc6-a8ef-6bfcd556dd54 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.523126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 232/461] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:13.528480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.528769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.529048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.531055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c465b6-4636-4e95-a981-066cb574b90e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:13.546241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:13.547462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:13.576804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819a833-2127-41b5-b780-730a56683017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:13.578318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a833-2127-41b5-b780-730a56683017 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.578619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a833-2127-41b5-b780-730a56683017 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.578902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4819a833-2127-41b5-b780-730a56683017 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.580316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a833-2127-41b5-b780-730a56683017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:13.580636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a833-2127-41b5-b780-730a56683017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:13.582469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4819a833-2127-41b5-b780-730a56683017 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:13.605665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:13.606681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 230/462] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:13.607534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819a833-2127-41b5-b780-730a56683017 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:13.624297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 233/463] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:13.632424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:13.719513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4819a833-2127-41b5-b780-730a56683017 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:13.744081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:13.748102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f1445-5094-4995-8203-2058ec3f34aa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:13.755535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:13.756471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.756763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.757041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.767615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:13.769086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:13.783064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:13.783806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.784364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.784624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.785306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a74d2540-e6d2-4be6-ba0f-661572bcc9e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:13.786524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:13.787512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:13.788509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:13.788509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:13.788509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:13.788509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:13.788714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:13.790940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 231/464] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:13.795484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d588-99ac-471a-b37d-d19ffb54de5b None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.795484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d588-99ac-471a-b37d-d19ffb54de5b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.795484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d588-99ac-471a-b37d-d19ffb54de5b None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.795484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 232/465] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:13.795484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1d588-99ac-471a-b37d-d19ffb54de5b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:13.893912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 234/466] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:24:13 2022] POST /identity/v3/users => generated 246 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:13.894271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1d588-99ac-471a-b37d-d19ffb54de5b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:13.895359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:13.949704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:13.951242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:13.951542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:13.951868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:13.958769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:13.972701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:13.974481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-cd1b2213-f871-4850-819d-04387e964a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:13.994761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:13.995633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:13.996822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:13.996822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:13.996822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:13.996822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:14.001535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 233/467] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:14.003026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.007515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.007515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.023409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:14.024824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:14.041429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571effe-9888-4cdc-8a44-d8a59206ea60 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.053243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:14.090266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.091261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.091725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.092180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.093129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:14.093635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.095254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.111036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 235/468] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:14.112236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.233590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d629c-ecdb-4161-9cad-149bfc22cf39 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.237607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:14.241907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 234/469] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:14.242833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.279297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.280136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.281826 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.282107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.282857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:14.283162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.284838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.304185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 236/470] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:14.305042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.377673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b0f9f-3546-412b-bfce-c241299efc13 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.406264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974b105-1047-4f82-8e67-8568b885a963 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.415596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.416536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.416958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.417474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.418306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.420373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.441902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.442699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.443042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.446469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.454592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.456082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:14.473484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:14.480252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:14.481094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:14.481094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:14.481094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:14.481094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:14.481094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 237/471] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:14.481094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.498624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:14.511581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 235/472] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:24:14 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:14.513016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.539429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:14.631712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6663df1-ff5a-4e0e-bf01-a85c53b668cd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.649053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e0852b-3fe2-4e6f-bde1-002fd8cdee4a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.673852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f2417-cee9-4147-88bf-d5771a598fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.674841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f2417-cee9-4147-88bf-d5771a598fe6 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.675212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f2417-cee9-4147-88bf-d5771a598fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.675496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f2417-cee9-4147-88bf-d5771a598fe6 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.676253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f2417-cee9-4147-88bf-d5771a598fe6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.678208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f2417-cee9-4147-88bf-d5771a598fe6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.695418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.703843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.703843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.703843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.703843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:14.703843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.703843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8466739-adcd-4005-b7f7-c4296246e787 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.712899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 236/473] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:14.714190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367559-e5c6-4ac5-8649-5eec8e6b578e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.714583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367559-e5c6-4ac5-8649-5eec8e6b578e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.714962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35367559-e5c6-4ac5-8649-5eec8e6b578e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.719852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 237/474] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:14.719852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367559-e5c6-4ac5-8649-5eec8e6b578e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.783261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 238/475] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:24:14 2022] POST /identity/v3/users => generated 248 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:14.785446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.785581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.785786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.809948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:14.811225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:14.813966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:14.860822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35367559-e5c6-4ac5-8649-5eec8e6b578e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:14.898086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:14.898944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:14.899309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:14.899712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:14.900556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:14.902497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:14.907055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 239/476] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:14.932313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:14.934607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:14.974929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 238/477] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:24:14 2022] POST /identity/v3/users => generated 249 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:24:15.066767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1ca4-ebc0-43c2-adb9-b98a47e29612 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:15.246159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:15.246830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:15.247109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:15.247380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:15.248164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:15.250075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:15.293869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.sql.core [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 20:24:15.293869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 20:24:15.293869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [parameters: {'id': 'd2eafc6c218f4f66b09b1330660ede40', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 20:24:15.293869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80393) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jun 06 20:24:15.310079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return ref.to_dict() Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 20:24:15.311484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application yield resource Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application next(self.gen) Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise self.value Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application session.commit() Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 20:24:15.313295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.session.flush() Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application compat.raise_( Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 20:24:15.314270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 20:24:15.315103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application conn.query(q) Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application result.read() Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 20:24:15.316020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [parameters: {'id': 'd2eafc6c218f4f66b09b1330660ede40', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:15.316821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 06 20:24:15.317768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:15.320471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 240/478] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 20:24:14 2022] POST /identity/v3/roles => generated 144 bytes in 380 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:15.323784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:15.470380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227a87-212b-4178-81d7-b6b8322bbd41 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:15.512293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:15.518380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6b2dc-d606-448c-ab72-11e1d7eb2672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:15.519188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b2dc-d606-448c-ab72-11e1d7eb2672 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:15.519471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b2dc-d606-448c-ab72-11e1d7eb2672 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:15.519773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6b2dc-d606-448c-ab72-11e1d7eb2672 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:15.530669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6b2dc-d606-448c-ab72-11e1d7eb2672 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-fff6b2dc-d606-448c-ab72-11e1d7eb2672 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:15.539225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:15.541808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:15.542821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 239/479] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:15.546981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:15.634938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:15.678441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7d23-dad6-4b17-8927-cd14c0263c2b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:15.726206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:15.726883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:15.727163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:15.727438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:15.728220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:15.728530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:15.729895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be7f457-e2e5-47d2-8489-08a8a53db317 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:15.743153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 241/480] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:16.569230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7fec-f2c4-4779-8d24-217baa7e5b5c None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:16.569708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7fec-f2c4-4779-8d24-217baa7e5b5c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:16.569708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7fec-f2c4-4779-8d24-217baa7e5b5c None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:16.571315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 240/481] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:16.586646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:16.587144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:16.587219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:16.604024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:16.605202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:16.672827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 242/482] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:16.721129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7fec-f2c4-4779-8d24-217baa7e5b5c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:16.857335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7fec-f2c4-4779-8d24-217baa7e5b5c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:16.907169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:16.907977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:16.908258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:16.908527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:16.926352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:16.933899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:16.934761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:16.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 241/483] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:16.941740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:17.043785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:17.108285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbca300-cb7d-4440-9f5d-6d12116004d9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:17.162254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:17.163019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.163303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.163578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.164364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:17.164668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:17.166032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ab22b-52ad-4d28-ae32-7d63be6890ee None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:17.178944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 243/484] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:17.183981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:17.285175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:17.348387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296bd4a9-d250-4bc7-b792-b6c1651261f4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:17.401803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:17.402467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.402745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.403011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.420915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:17.422995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:17.449049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:17.449950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:17.451344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:17.451344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:17.451344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:17.451344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:17.451344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 242/485] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:17.456025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:17.456177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ae16c-5391-4815-84ce-e6a85973d95a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.456467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ae16c-5391-4815-84ce-e6a85973d95a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.456633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ae16c-5391-4815-84ce-e6a85973d95a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.458225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 244/486] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:17.463910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.464201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.464478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.495343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:17.496548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:17.605927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 245/487] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:17.630220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:17.685939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:17.690022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2770914b-c5a7-4417-a1bf-506f27e2ed52 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:17.691773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:17.758753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:17.759718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.760319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.761752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.762688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:17.763297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:17.765628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:17.780420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da985f-f7d4-4e02-b0f2-a215d0bd7a29 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:17.792705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 243/488] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:17.799954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:17.841823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4a58c9-cf55-41b9-9b32-be1434d32a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:17.842630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a58c9-cf55-41b9-9b32-be1434d32a25 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.843195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a58c9-cf55-41b9-9b32-be1434d32a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.843490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4a58c9-cf55-41b9-9b32-be1434d32a25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.857034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4a58c9-cf55-41b9-9b32-be1434d32a25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fc4a58c9-cf55-41b9-9b32-be1434d32a25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:17.868495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:17.869326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:17.870103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:17.870687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 246/489] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:17.872050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6cc5-5510-4a27-94fc-b00d0b7c10d8 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.872339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6cc5-5510-4a27-94fc-b00d0b7c10d8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.872618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe6cc5-5510-4a27-94fc-b00d0b7c10d8 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.875128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 247/490] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:17.877021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e04d63-8d79-41f9-b48c-219e34d7d39f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.877311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e04d63-8d79-41f9-b48c-219e34d7d39f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.877592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e04d63-8d79-41f9-b48c-219e34d7d39f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.879167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 248/491] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:17.879648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e04d63-8d79-41f9-b48c-219e34d7d39f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:17.926946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688e881-e0ae-454d-94be-aa6efc6ac707 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:17.963741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:17.964978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.965437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.965721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:17.974208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:17.975645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:17.987643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e04d63-8d79-41f9-b48c-219e34d7d39f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-76d25891-e350-471d-9cb5-f06bf946dfd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:17.991834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:17.992694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:17.993580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:17.997510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 244/492] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:24:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:17.998736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:17.999027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:17.999303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.000144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.015740 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:18.019801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:18.024384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.025458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.025749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.026027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.027146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:18.027462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.029236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90737f8d-e238-4d8b-9c48-d22927d18c54 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.034816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.040231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 249/493] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:18.041688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.042114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.042517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.059374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:18.060634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:18.094490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 245/494] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:18.095395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.140936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 250/495] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:18.142054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.224494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1ea3-1a75-4007-b64c-a08704553cd5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.256883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7bf246-7330-4b3a-bcda-d13b585e83cd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.258463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.261162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.261843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.262124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.262395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.263223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:18.263533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.265035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.292650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.294168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.294630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.295092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.301752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 246/496] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:18.305997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.306991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.308128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.319398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:18.328249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:18.341538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:18.342347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:18.342347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:18.342347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:18.342347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:18.342347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 251/497] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:18.342347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.421044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a779cf-4085-4aa0-ab94-2bc4bc2501d7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.443080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3c4b1-d301-4f1c-83ed-bf367ed6dc6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.459298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.460217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.460418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.460814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.471380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:18.478957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:18.479863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:18.480657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:18.481245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.481245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.481245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.481439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 247/498] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:18.481507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.481572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.493503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:18.501891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:18.502765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:18.503544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 252/499] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:18.504977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.523874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.564376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.596472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f044ce-7d76-4d64-b2aa-2c11654001e1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.616256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b67b1d2-68ae-4339-bcba-dff4eb347177 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.633295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.633944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.634235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.634513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.654083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.654735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.655022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.655296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.656062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:18.656370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.657731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.677039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 253/500] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:18.677039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.710465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.712071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.751611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.774605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2de5a-004c-4541-a549-8bd027062120 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.777822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.809017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 248/501] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:18.811692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.811846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.811846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.811846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.811846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.812192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:18.812289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.813576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.822820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 254/502] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:18.823840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.911571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84190c6-8b40-42bd-8263-34b272239c46 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.922703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b9756-4a5b-4837-8fee-33a5a0f941a8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:18.945227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.945888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.946175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.946454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.947191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:18.947500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.948898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfddf0a3-44e1-4607-a7b3-1b4619ee1f43 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.955798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:18.956575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 249/503] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:18.956716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.956871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.957128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.957864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6754b2f-2133-472e-aa8c-989fe7a4859a None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:18.958131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6754b2f-2133-472e-aa8c-989fe7a4859a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:18.958418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6754b2f-2133-472e-aa8c-989fe7a4859a None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:18.959818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 250/504] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:18.960418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6754b2f-2133-472e-aa8c-989fe7a4859a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.962051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:18.967960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:18.969987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:18.985763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:18.986618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:18.987504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 255/505] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:18.988092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:18.992842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.060711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6754b2f-2133-472e-aa8c-989fe7a4859a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.087551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2114f-bb07-4ea4-8019-9d24eb27438f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.092726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.094045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.094331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.094607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.104892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.106327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.121396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.122172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.122454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.122731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:19.123306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:19.124324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:19.130098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:19.130098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:19.130098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.130098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:19.130098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 251/506] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:19.130098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.133849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.135296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5edd1280-80a0-473f-905a-70991d67b4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:19.152989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:19.153992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:19.155026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:19.155026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:19.155026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.155026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:19.155026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 256/507] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:19.155374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.155728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.156512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.172583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:19.173757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:19.213351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.215533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.230618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c887-566f-4b8e-b841-d31b79924a24 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.238418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 257/508] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:19.239251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.270121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.271089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.271405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.271751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.283056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.284802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:19.301795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:19.303646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:19.304516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:19.304516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:19.304516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.304516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:19.304516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 252/509] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:19.304516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.341690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe804a7c-ac02-4261-b600-ff6de9744879 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.375349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.376038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.376321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.376596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.377414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:19.377722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.379134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.403167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ba2cc-3287-4595-880a-3c1c700f6358 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.415132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 258/510] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:19.415976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.436746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.437430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.437712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.437989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.438701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:19.439003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.440402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.442594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.449943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.452022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 253/511] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:19.461482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.514145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25215596-c129-4fcb-96a1-7ae6798609fc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.547178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.547846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.548159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.548434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.549151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:19.549456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.550540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9cc3a9-0b5d-44b2-979d-6a3643b7d61e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.550817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.562688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 259/512] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:19.563471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.584716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.585495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.585780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.586058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.595895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.597285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:19.615093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:19.616085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:19.621867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:19.621867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:19.621867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:19.621867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:19.621867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 254/513] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:19.621867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.653694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.659566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b7fff-cff1-40f5-95b5-6e49b79e009a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.664387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.692807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.693613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.693907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.694190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.694908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:19.695217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.696607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.708428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 260/514] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:19.709303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.713464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2077ba0b-12fb-4784-9168-06ac24d0a4ca None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.746875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.747515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.747833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.748116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.805040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa4fd19-f85c-487f-bbb2-03521f938c83 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.813305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.814747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.838600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.839461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.839777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:19.840060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:19.846580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:19.847973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:19.861627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:19.863023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:19.863946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:19.864777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 261/515] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:19.865253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.883717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:19.913013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 255/516] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:19.913830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:19.965019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf830a1-92ba-423a-9ca8-ec3439e49e62 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:19.998335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:19.999117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:19.999448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.000238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.007036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.008523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.010207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953a561-06a7-4bad-ae80-bf155cb97c72 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:20.022615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:20.023703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 262/517] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:20.024581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.047809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.048479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.048768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.049044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.049754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:20.050061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.051420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.062874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 256/518] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:20.063646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.075349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.098481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.120630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f207890-5d21-4f4a-b55f-36cbc3ffda00 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.153902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.154687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.154968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.155233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.161628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e8f0b-dba6-482f-b9a1-53398aec6e87 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.162246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.163699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:20.178299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:20.179335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 263/519] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:20.184890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.195041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.195718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.196007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.196289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.197068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:20.197379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.198803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.215624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 257/520] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:20.216427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.278753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c811-e7df-4c65-b292-648b8bb8c60c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.286920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.313264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.314043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.314324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.314610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.317263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.320046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88138552-5b32-4754-93b1-47ceed9b0c88 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.324567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.326003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:20.343021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:20.344430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:20.345342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:20.345342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:20.345342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:20.345342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:20.345342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 264/521] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:20.345607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.361018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.361018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.361018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.361018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.361385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:20.361385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.362819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.380687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 258/522] 10.222.0.41 () {56 vars in 1139 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:20.381476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.442499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b031f-fda4-4496-a7e9-ed8fd3114992 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.475514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.476199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.476482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.476741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db881e-50c2-48ad-ae38-df7192910d79 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.476844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.477439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.478869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.502173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.510459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.511106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.511388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.511677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.512497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:20.512803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.514202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.530477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 259/523] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:20.531873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.561903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 265/524] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:24:20 2022] POST /identity/v3/users => generated 247 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:20.562708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.614894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.651818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf931794-6ac9-4019-bf34-558b129ec38c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.680472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afe167-f191-4d35-aeb4-5203eb7314a9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.689580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.690246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.690528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.690808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.691646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=f310f6d84a3d42df9d473677ac5b6557) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:20.691987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.693450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.703828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 260/525] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/role_assignments?user.id=f310f6d84a3d42df9d473677ac5b6557&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:20.703828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.728096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.728791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.729120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.729346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.730115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:20.730422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.732037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:20.746923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 266/526] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:20.747854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:20.773622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.836723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fc03e-c9a9-4088-a83c-3c94c0f8422b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.867271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c67949-d567-4fda-90ec-2745acafb39f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:20.870006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:20.882484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.883178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.883429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.883737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.914145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:20.914797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:20.915082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:20.915355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:20.963802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:20.965385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.003333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.005043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.025131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.099810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 261/527] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:21.100426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.122583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 267/528] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:21.123618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.126914 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.215751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2883db-21d2-417d-89d5-68a0fc7a1706 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.231175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d9f9d-607e-48ef-ab2d-b5d4f10e7c80 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.262975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.264552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.264949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.265829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.270617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.271589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.272122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.272539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.280505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.280505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.289726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.291441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:21.292853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:21.293684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:21.294521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:21.298834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 262/529] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:21.299706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.312490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:21.318459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:21.319331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:21.320176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:21.320176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:21.320176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:21.320176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:21.323746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 268/530] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:21.324649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.386256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.433882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4060fe02-870a-4368-968c-91d8d6852055 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.437660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433958dc-0bd5-4cf9-bcb8-5c36ecbc2fc5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.473254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.474080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.474490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.474919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.475932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:21.476341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.478073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.480522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.481316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.481603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.481880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.494115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.496179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.503977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 269/531] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:21.505090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:21.516398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:21.517785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:21.519493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:21.519493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:21.519493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:21.519493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:21.519493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 263/532] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:21.523979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.602928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.623887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ffc18-0bdf-416e-ac55-9ff91c6e219a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.642426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.657422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b0f52c-0ca3-481f-80d4-3431940be163 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.665608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.666444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.666877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.667646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.668574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:21.669353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.671246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.691350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 270/533] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:21.691350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.699576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.700551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.701542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.702029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.704266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=bb84e744681149aea78fcc984605bf33) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:21.704791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.707201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.715087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 264/534] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/role_assignments?user.id=bb84e744681149aea78fcc984605bf33&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:21.716298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.841139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e61d25d-925e-44bf-9b5d-ed9bafd3170c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.871643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:21.886156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.887553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.888037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.888482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.894082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2c5b83-3665-4e81-b7a9-7faa5be7e697 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:21.898602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:21.900401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:21.918802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:21.919778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:21.920605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:21.923432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 265/535] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:21.933293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:21.996449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:21.997133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:21.997419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:21.997688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:21.998596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:21.998708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:22.000691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:22.033732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 271/536] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:22.036513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:22.087536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c405d-db27-4f06-a4d3-2d782be6d23d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:22.104431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:22.161297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:22.163093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:22.163943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:22.165366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:22.167550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:22.168226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:22.170602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:22.196122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 266/537] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:22.200577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:22.233539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f11bb55-d7c6-48f3-8e0c-e74637a9d8b4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:22.292280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:22.307854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:22.308655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:22.308936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:22.309208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:22.327848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:22.327848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:22.368996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:22.369931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:22.370748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:22.375685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 272/538] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:22.377715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:22.410062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d9d83-158f-406f-97a8-e5b9dd30caa0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:22.478377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:22.479039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:22.479322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:22.479601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:22.480450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=e527c9fc7ae1475595c311ac3215b51f) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:22.480761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:22.482193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:22.496743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 267/539] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:22 2022] GET /identity/v3/role_assignments?user.id=e527c9fc7ae1475595c311ac3215b51f&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:22.553623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:22.562764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc34f7-2b1c-4a1d-a891-531e371cef79 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:22.599169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:22.599857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:22.600140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:22.600435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:22.601243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:22.601783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:22.603550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecb5e6-d23f-4fad-a06e-a755bd3c52d8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:22.618580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 273/540] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:22.624674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:22.749903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd59a-82cc-42b6-b060-ccd3cb6df2ad None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:22.759114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:22.784953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:22.785666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:22.785955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:22.786228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:22.787056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=9110e7fab9084faa94b5d345bb9d519a) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:22.787362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:22.788838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822320a3-8da2-4696-894f-8decd9677c2e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:22.795077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 268/541] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:22 2022] GET /identity/v3/role_assignments?user.id=9110e7fab9084faa94b5d345bb9d519a&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:23.056789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab546ab-d463-419e-89bf-e0b0031c9c4a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.057300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab546ab-d463-419e-89bf-e0b0031c9c4a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.057300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab546ab-d463-419e-89bf-e0b0031c9c4a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.058936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 274/542] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:23.064605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834548-c29b-4883-a98d-c4e275adde10 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.065071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834548-c29b-4883-a98d-c4e275adde10 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.065167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77834548-c29b-4883-a98d-c4e275adde10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.087714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-77834548-c29b-4883-a98d-c4e275adde10 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:23.088916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-77834548-c29b-4883-a98d-c4e275adde10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:23.194449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 269/543] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:23.219246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab546ab-d463-419e-89bf-e0b0031c9c4a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:23.244622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1db5-c370-4322-a0bd-5b5f4def0c95 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.244919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1db5-c370-4322-a0bd-5b5f4def0c95 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.245179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f1db5-c370-4322-a0bd-5b5f4def0c95 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.246820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 270/544] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:23.251782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.252068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.252345 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.269113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:23.270302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:23.350944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 271/545] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:23.402196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab546ab-d463-419e-89bf-e0b0031c9c4a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:23.422595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:23.454324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:23.455120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.455396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.455683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.472420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:23.475751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:23.484289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:23.488958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:23.489724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 275/546] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:23.492257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:23.536021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:23.603061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8dbf8-e355-427e-aa28-097afaf3df03 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:23.650309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed2183-5a6b-4dd5-a676-e43ab63c05e4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:23.686182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:23.687388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.687691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.687995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.688740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:23.689045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:23.690603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:23.700165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 276/547] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:23.706122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:23.728312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:23.729111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.729394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.729667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.750794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:23.763284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:23.764206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:23.765958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 272/548] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:23.771032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:23.848589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:23.909950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79f26b-3eea-436a-b34c-af9ad52bfe19 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:23.919571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b873e8-ce19-4920-89c6-903e7051f3f0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:23.927685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:23.957151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:23.957954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.958337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.958710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:23.959656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:23.960112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:23.962110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:23.982957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 273/549] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:23.991685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:23.991838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:23.991838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:23.991838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:23.991838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.019276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.020762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:24.048792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:24.049667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:24.050509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:24.050509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:24.050509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:24.050509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:24.062721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 277/550] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:24.063597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.182156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e31c6b-3804-4387-9193-c0f808321013 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.201802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8099838-e2f0-4d42-bd5f-b2cccb0a569d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.213159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:24.243940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.243940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.243940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.243940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.244452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:24.244452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.244878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.267626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 278/551] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:24.277617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.289777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.290580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.290861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.291133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.310071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.311533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:24.337799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:24.338744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:24.339608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:24.339608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:24.339608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:24.339608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:24.340295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 274/552] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:24.349097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.363105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:24.420548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cc2a8-ac95-4c16-8492-7f4cda3b1750 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.467462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.468235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.468546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.468801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.486189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.489794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:24.510879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:24.511889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:24.512783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:24.516852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 279/553] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:24.518898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.589568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6e6b0-abab-405b-80c7-d3d6864663e7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.592056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:24.627549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.628265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.628551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.628828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.630818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:24.631137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.632788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361fb00-6874-464e-b455-649eafedc9cc None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.636389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2dd309-6956-4547-a5f5-b062f3461a86 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.646395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 275/554] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:24.647861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b06cad-2f27-4a6e-b4cf-3aa376143ee9 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.648171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b06cad-2f27-4a6e-b4cf-3aa376143ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.648528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b06cad-2f27-4a6e-b4cf-3aa376143ee9 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.649951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 276/555] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:24.654907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b06cad-2f27-4a6e-b4cf-3aa376143ee9 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.676822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.677661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.678042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.678420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.679348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:24.679806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.681327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc326a3e-549d-4665-af8c-91e464c11bb4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.699745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 280/556] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:24.701147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.701533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.701912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.719190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:24.720472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:24.744040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:24.788115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b06cad-2f27-4a6e-b4cf-3aa376143ee9 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.791349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 281/557] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:24.797310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.827585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.828406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.828924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.829388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:24.838782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:24.840649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:24.859371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:24.861978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:24.863356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:24.864312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 277/558] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:24.865755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:24.946062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e1df6e-74ed-47c2-b65b-60dfe0daaf5a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.978511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8ab20-1584-4525-8b4b-e9e0b758f4ef None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:24.985204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ae7de-8cf1-4a44-bfe9-c7700fc02bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:24.985856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ae7de-8cf1-4a44-bfe9-c7700fc02bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:24.986138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ae7de-8cf1-4a44-bfe9-c7700fc02bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:24.986414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ae7de-8cf1-4a44-bfe9-c7700fc02bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.005832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.015347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.020320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.020320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.020320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.028204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:25.038667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:25.039870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:25.040777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 278/559] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:25.041579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.069949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ae7de-8cf1-4a44-bfe9-c7700fc02bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.072478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ae7de-8cf1-4a44-bfe9-c7700fc02bc5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.114678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.151249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a29394-b90c-4eb9-9e48-c82a71d4637b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.185482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 282/560] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:25.185482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04273254-905e-4db5-b23f-27ddb9e2f657 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.185482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04273254-905e-4db5-b23f-27ddb9e2f657 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.185482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04273254-905e-4db5-b23f-27ddb9e2f657 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.185482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 283/561] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:25.185482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04273254-905e-4db5-b23f-27ddb9e2f657 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.192803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.193462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.193748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.194035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.194866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:25.195176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.196646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.219827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 279/562] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:25.219827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.282275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.314102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04273254-905e-4db5-b23f-27ddb9e2f657 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.347008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02a865-fcfa-4a38-952d-eb3f26e317f8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.348342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.349106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.349425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.349705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.350473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:25.350779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.352171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af4679d-1aa4-40cf-a598-08f8b6fb3380 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.357219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.365749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 284/563] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:25.365749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.365749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.365749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.382327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:25.383284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:25.390034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.390877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.391165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.391439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.402339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.403835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:25.422481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:25.423396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:25.424299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:25.424299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:25.424299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:25.424299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:25.425560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 280/564] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:25.426584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.453458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 285/565] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:25.454299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.536679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.542615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee583038-eb4c-44cd-8ae8-62a151a15b85 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.564963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fcc882-c806-4e30-9aff-f5d07e079ab9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.577996 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.578784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.579070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.579347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.589123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.590566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.592170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.598363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.603832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.603832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.603832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:25.609307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:25.610187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:25.611059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:25.611059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:25.611059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:25.611059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:25.611288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.611358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 281/566] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:25.611358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.611988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:25.630717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:25.631629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:25.632562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:25.632562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:25.632562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:25.632562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:25.632562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 286/567] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:25.632854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.709611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de5471-c7b6-4d88-bf16-6820fc6287bf None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.731841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29644e5-5d78-45dd-9d85-7dc76b55e757 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.743205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.743895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.744180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.744462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.745181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:25.745489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.746871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c4a0b-2101-49b0-bd41-effeb174a14f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.757708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 282/568] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:25.758912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67fdb-a326-4607-81a1-e1753fdfd7a0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.759202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67fdb-a326-4607-81a1-e1753fdfd7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.759480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa67fdb-a326-4607-81a1-e1753fdfd7a0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.760853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 283/569] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:25.761421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67fdb-a326-4607-81a1-e1753fdfd7a0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.768783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.769567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.769854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.770130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.772234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.781084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:25.798203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:25.799360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 287/570] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:25.800444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.834172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:25.868801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa67fdb-a326-4607-81a1-e1753fdfd7a0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.889116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b509ab8-8919-49ea-9079-610df0e1ed64 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:25.900516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.901162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.901442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.901714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.902416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:25.902714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.904126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.917354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 284/571] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:25.918313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:25.926823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:25.927475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.927797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.928421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.929139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:25.929767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:25.931131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167bd34-92ad-49d6-8e9d-4322091ee44d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:25.943115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 288/572] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:25.944393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:25.944681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:25.944956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:25.961193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:25.962353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:25.986385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.019059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfd278-150b-4b87-8f40-b7fe2e3a58ab None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.026796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 289/573] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:26.027500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.054012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.055014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.055297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.055569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.057657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.062108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.063572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:26.080452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:26.081613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 285/574] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:26.083023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.130906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193059e-3dc5-41e6-8e06-b78e07889394 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.164477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.165135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.165415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.165691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.166429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.166744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.168204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.175946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 290/575] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.176716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.181036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eab8c-e587-4a43-8338-fcde790ea3e9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.206570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.213918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.221038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.221038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.221038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.222323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.223788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:26.238708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:26.239597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 286/576] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:26.241275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.277019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.283092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad30a06-21b0-4c7a-9f47-b8377687913f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.315201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.315944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.316226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.316498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.323758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.325181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.336969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c904a1-dffd-479a-8275-51c98db9990e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:26.339476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:26.340429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:26.341364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 291/577] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:26.342597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.368919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.369760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.370045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.370371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.381968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:26.389661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:26.391222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:26.392100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 287/578] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:26.392801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.439385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.442496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0d480-4c13-4fc2-92fb-8d87d410f481 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.475365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.476071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.476356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.476633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.477415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.477721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.479145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.488946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7616b2-0ccb-4b9b-8fab-5e1505c888ef None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.491497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 292/579] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.492307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.507535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.525459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.526239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.526522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.526796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.537470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.538944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:26.555485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:26.556641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:26.557509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:26.557509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:26.557509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:26.557509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:26.557509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 288/580] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:26.557509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.594947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7cb02-9187-438b-bfc9-b03a146fe33f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.628410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.629058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.629340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.629614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.630392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.630697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.632155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.643191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 293/581] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.643985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.653245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.656391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea6a30a-c27a-4e62-809f-647f0afec3c8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.690798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.691599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.691906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.692179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.692993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.693585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.694703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.710983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 289/582] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.711800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.720744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.744818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c566b6-8287-407b-a604-44a8ff75fbed None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.778052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.778706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.778983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.779256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.780013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.780315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.781692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.789508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 294/583] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.790271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.812402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2481fa-22fa-4e25-a89f-d9beef329112 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.844574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.845277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.845508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.845780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.846590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=e168d14c0eac4ee2a5acdc23c39118d9) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.846898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.848359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.853663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 290/584] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/role_assignments?user.id=e168d14c0eac4ee2a5acdc23c39118d9&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.854398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.894072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.898139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e271d8a-2672-419e-93dc-2cb2b1f99364 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:26.938916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:26.939594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:26.940072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:26.940350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:26.941235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:26.941544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:26.942964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:26.946033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:26.956750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 295/585] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:26.958581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:26.983427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d06de3-e703-457a-9b6e-106768170134 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.021598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.022639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.023037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.023414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/f310f6d84a3d42df9d473677ac5b6557/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.041322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=f310f6d84a3d42df9d473677ac5b6557, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.042940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.067205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213226-f00b-4610-9e08-795199e63b51 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.100618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.101340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.101623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.101900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.112321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.113754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:27.130782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:27.132334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:27.133676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:27.133676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:27.133676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:27.133676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:27.143838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 291/586] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:26 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/f310f6d84a3d42df9d473677ac5b6557/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:27.143838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.144123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 296/587] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:27.144123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.208904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:27.233239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:27.271335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900335ec-7452-4ebc-85b9-45d6cb8105c2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.290314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11061a-3374-4abe-b77b-f8d58dcd189e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.310750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.312251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.312617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.312965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.326427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.328952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.338287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.339199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.339557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.339932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.349246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:27.350675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:27.351757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:27.353109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:27.353109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:27.353109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:27.353109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:27.353389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.356292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 292/588] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:27.356292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:27.373857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:27.376905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 297/589] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:27.379264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.481898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be1914-c0b1-492a-86b1-9375679412a1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.490502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9317-0432-4e04-8158-5434631b573c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.492153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:27.518805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.519863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.520230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:27.520946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.521429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.522351 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:27.522877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.524576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.532995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.533682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.534712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.534712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.536372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:27.536695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.538390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.539587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 298/590] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:27.540533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.556898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 293/591] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:27.557858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.665090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2db9e7-8ca1-4148-a69e-ba55f895a5be None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.685468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8ddf4-e6fa-43d7-b6ac-b0a4f45d6f5b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.701718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.702700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.703044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.703408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.711502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.713077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.727505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.728217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:27.728498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:27.730291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:24:27.732652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:27.738070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.738070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.738070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:27.738070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.738380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 294/592] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:27.738468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.738550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.760747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 299/593] 10.222.0.41 () {56 vars in 1139 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:27.762013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.794531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:27.797986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:27.851043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4adf0-357f-40d2-a5da-9d8f48b8e970 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.873847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b118715-a40c-460b-9b70-4bf9c52d8f70 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:27.888335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.889392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.889884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.890313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.897829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.899732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.910363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:27.911031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:27.911316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:27.911594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:27.912473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:27.912780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:27.914194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:27.917406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:27.918404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:27.919438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:27.921846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 295/594] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:27.922824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:27.933752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 300/595] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:27.934706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.024539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.032386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95689909-710a-4696-9e5e-d6a82da154c5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.035280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.042845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a1f9-8384-4de6-8d43-72d21e555592 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.070106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5146180-12aa-432c-b198-ca001364550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.070899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5146180-12aa-432c-b198-ca001364550b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.071204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5146180-12aa-432c-b198-ca001364550b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.071725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5146180-12aa-432c-b198-ca001364550b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.079751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.080643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.081964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.082450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.083412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:28.083609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5146180-12aa-432c-b198-ca001364550b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.084118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.085090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5146180-12aa-432c-b198-ca001364550b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.086145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.099819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 301/596] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:28.100981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d5146180-12aa-432c-b198-ca001364550b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:28.104519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:28.105392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:28.106248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:28.106248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:28.106248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:28.106248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:28.110475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 296/597] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:28.111290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5146180-12aa-432c-b198-ca001364550b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.211144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f07ff4-b068-4c8c-8ce7-0f2a6caf5d78 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.217089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5146180-12aa-432c-b198-ca001364550b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.248495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.249731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.250234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.250709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.257532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.258406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.258792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.259390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/bb84e744681149aea78fcc984605bf33/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.266563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.268568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.271873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.278244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=bb84e744681149aea78fcc984605bf33, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.282344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:28.289813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:28.290770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:28.292266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:28.292266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:28.292266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:28.292266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:28.292266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 302/598] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:28.292923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.341762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 297/599] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:28 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/bb84e744681149aea78fcc984605bf33/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:28.343990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.356069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.405189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26f8d7-b7ed-4e97-a503-5bc267a53e07 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.444580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.445576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.445998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.446447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.447334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:28.447836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.449500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.463985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068367-8423-43b4-9794-59920eaa4a29 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.465303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 303/600] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:28.475967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.507258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.508171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.508548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.508968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.509723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:28.510241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.511596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.524598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 298/601] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:28.533494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.548108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.631539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839dddb-e28c-4231-805e-7f718001fc79 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.672593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.674686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.675795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.676082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.676355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.684873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.687253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:28.705907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:28.706840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:28.707732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:28.708319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 304/602] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:28.712630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8ce41-1e53-4cef-ba34-bda74a8db32b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.714899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.758812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.759963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.760249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.760527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.770158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.772109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:28.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:28.803874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:28.804890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:28.809754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 299/603] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:28.816520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:28.881863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb0f06-9322-4a98-b1b5-db3941f82563 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.912769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:28.966068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7adc3c9-cc64-470d-b987-a8b846d79f4e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:28.980367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:28.981330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:28.981330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:28.981584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:28.986321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:28.986321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:28.986321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:29.000746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:29.016932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 305/604] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:29.019615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:29.026694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:29.027363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.027646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.027942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.028724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:29.029030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:29.035608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:29.049707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 300/605] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:29.057549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:29.175402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16159e8-5de5-459b-bc69-b64a7cfb3fd5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:29.213152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa89dc-b3b9-4e9c-96e2-af3f0a077408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:29.214029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa89dc-b3b9-4e9c-96e2-af3f0a077408 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.214307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa89dc-b3b9-4e9c-96e2-af3f0a077408 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.214577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa89dc-b3b9-4e9c-96e2-af3f0a077408 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/e527c9fc7ae1475595c311ac3215b51f/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.241463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa89dc-b3b9-4e9c-96e2-af3f0a077408 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=e527c9fc7ae1475595c311ac3215b51f, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:29.243504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa89dc-b3b9-4e9c-96e2-af3f0a077408 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:29.256301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c6516-9d63-49e6-b016-a12fc8b34963 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:29.293278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b69b90-12eb-4d16-8193-509229f38347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:29.294162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69b90-12eb-4d16-8193-509229f38347 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.294442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69b90-12eb-4d16-8193-509229f38347 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.294710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b69b90-12eb-4d16-8193-509229f38347 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/9110e7fab9084faa94b5d345bb9d519a/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.310928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b69b90-12eb-4d16-8193-509229f38347 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=9110e7fab9084faa94b5d345bb9d519a, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:29.312497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b69b90-12eb-4d16-8193-509229f38347 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:29.339849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 306/606] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:29 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/e527c9fc7ae1475595c311ac3215b51f/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:29.391335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 301/607] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:29 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/9110e7fab9084faa94b5d345bb9d519a/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:24:29.432049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de32c9-c292-4f84-b98e-ffe0db6d2751 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.432400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de32c9-c292-4f84-b98e-ffe0db6d2751 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.432601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de32c9-c292-4f84-b98e-ffe0db6d2751 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.434404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 307/608] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:29.447996 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3abb24-6fa8-4253-a59d-1e7fd6bcc5b3 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.448332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.448449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3abb24-6fa8-4253-a59d-1e7fd6bcc5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.448534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.448606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3abb24-6fa8-4253-a59d-1e7fd6bcc5b3 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.448802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.450248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 302/609] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:29.455981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.456290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.456537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.474663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:29.475976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:29.476353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:29.477494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:29.505541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:29.542625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:29.568353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 308/610] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:29.607764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 303/611] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:29.608282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:29.647384 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:29.739828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38939819-3aba-48de-ae24-785b2b9d7f15 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:29.766841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8514223-132f-41c1-8a05-e4f69849243a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:29.786223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:29.787385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.787677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.787961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.801753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:29.804994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:29.805789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:29.806075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:29.806354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:29.810040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:29.812054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:29.813198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 309/612] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:29.820685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:29.824607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:29.829380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:29.831535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:29.832839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 304/613] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:29.839913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:29.860235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:29.899050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:30.005461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9699c3-16a4-49d3-8a6f-7265aa5f89ee None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.040160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.040769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.041058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.041340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.042092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:30.042404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.042582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e673413-6168-4397-9172-501f7f494fe3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.043853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.053056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 305/614] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:30.062107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.112574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.113501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.115977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.116366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.117238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:30.117643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.119126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.137473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 310/615] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:30.145344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.196052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:30.220154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3655193-d614-4de8-a9e2-a3f0c2eeb015 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.264010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:30.288847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.289687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.289974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.290302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.302853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb169b-8feb-4676-987b-fffe62cf77c3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.314516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.316057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:30.343688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:30.344625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:30.345482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:30.345482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:30.345482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:30.345482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:30.345482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 306/616] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:30.347024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.371427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.372290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.372564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.372834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.394097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.396395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:30.428921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:30.429879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:30.430913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:30.430913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:30.430913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:30.430913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:30.430913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 311/617] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:30.437974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.516028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4445bf-7e8c-4681-9ea0-666abc2f7f5a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.552849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.553480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.553768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.554043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.554754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:30.555060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.556462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.569588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 307/618] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:30.570104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:30.574943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.677856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a5406-ed5a-4175-8502-d97215ed02e5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.714681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.715477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.715805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.716083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.724185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.725651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:30.742132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:30.743101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:30.744058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 308/619] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:30.749515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.825702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75efcfcc-e725-4506-9144-b6060f54c7e5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.826334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:30.864917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e4e5e4-9860-4af2-ab8a-f7e128702d88 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:30.894147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:30.902893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958a98-0279-4533-b203-67b141445c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.903571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958a98-0279-4533-b203-67b141445c3c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.903901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958a98-0279-4533-b203-67b141445c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.904176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16958a98-0279-4533-b203-67b141445c3c None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.905012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958a98-0279-4533-b203-67b141445c3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:30.905865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958a98-0279-4533-b203-67b141445c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.907394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16958a98-0279-4533-b203-67b141445c3c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.908338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:30.909274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:30.909557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:30.909832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:30.910562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:30.910870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:30.912267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:30.932138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 309/620] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:30.933959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958a98-0279-4533-b203-67b141445c3c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:30.938225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 312/621] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 501 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:30.943641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.074422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16958a98-0279-4533-b203-67b141445c3c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.108107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.123588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230cadf-9a5d-4c5a-ac99-d8b2e9612074 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.139324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6c623-bb3c-465c-b98c-1b1063e15dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.140957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6c623-bb3c-465c-b98c-1b1063e15dff None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.141440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6c623-bb3c-465c-b98c-1b1063e15dff None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.141928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6c623-bb3c-465c-b98c-1b1063e15dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.164838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.165630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.166009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.166456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.175926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.177532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a6a07a3f-d9c3-4bcb-925b-179bd597c005 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:31.195431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:31.196425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:31.198025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 313/622] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:31.198737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2b763-8dfe-4106-9728-a19f2ea74212 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.198737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2b763-8dfe-4106-9728-a19f2ea74212 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.198876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c2b763-8dfe-4106-9728-a19f2ea74212 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.200539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 314/623] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:31.202779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2b763-8dfe-4106-9728-a19f2ea74212 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.225014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6c623-bb3c-465c-b98c-1b1063e15dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.227286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6c623-bb3c-465c-b98c-1b1063e15dff None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.262922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.315945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c2b763-8dfe-4106-9728-a19f2ea74212 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.352089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 310/624] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:31.353998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.354471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.354929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.360331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.360950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.361234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.361506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.362718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:31.362718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.364091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.373396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:31.374872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:31.382352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 315/625] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:31.386996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.394890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.451522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 311/626] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:31.453303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.500100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c703c-a960-4ffb-b261-5e2749ba8afb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.520024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.541294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.542084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.542364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.542639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.556456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.558430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.573346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc83e3-7a18-4367-97b3-a00bf6147206 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:31.580934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:31.581831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:31.582694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:31.582694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:31.582694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:31.582694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:31.585034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 316/627] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:31.586053 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.623949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.626295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.626580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.626853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.644850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.646798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.659413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:31.672120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:31.673923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:31.674886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:31.674886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:31.674886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:31.674886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:31.678532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 312/628] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:31.679556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.707151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df891e0-eba0-4897-b105-13e8c5fadd6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.746770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.748079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.748606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.749649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.761183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:31.770567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:31.771532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:31.772408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:31.775313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 317/629] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:31.776471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.802605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.805111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04aee-ad3c-4dff-95af-a76a54848766 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.846717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.848510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.849010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.849523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.850628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:31.851161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.852891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.866416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 313/630] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:31.867595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.905790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed0bece-fe0c-4af1-bf92-fa1672b217f1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:31.941146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:31.944997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:31.945902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:31.946392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:31.946872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:31.947951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:31.948476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:31.950151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:31.963775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 318/631] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:31.964936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:31.987207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72028389-35b4-4db1-bca8-fae28977480e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.024968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.026517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.027035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.027714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.029276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:32.029795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.031671 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ee31b5-f92c-40b6-ae64-e3d71a248fa7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.043012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 314/632] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:32.045263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd14c26-b192-460c-8c0f-e7c8c0edfc82 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.045792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd14c26-b192-460c-8c0f-e7c8c0edfc82 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.047033 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd14c26-b192-460c-8c0f-e7c8c0edfc82 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.047768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.049218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 315/633] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:32.049945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd14c26-b192-460c-8c0f-e7c8c0edfc82 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.071430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce835d8-1b77-4a08-bdec-cad213ce2bca None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.109973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.110860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.111143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.111523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.118586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.120055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:32.135763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:32.136712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:32.137609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 319/634] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:32.138092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.158625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd14c26-b192-460c-8c0f-e7c8c0edfc82 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.180867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.193241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.194025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.194310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.194587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.201174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.202634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-de3d7e92-dfe7-4d4e-9540-75c2286f1285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:32.216877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:32.218451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:32.219344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 316/635] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:32.220014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.220304 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.220579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.237159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:32.238219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:32.240421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20983b0-7f56-425f-93f2-b785ea575771 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.272870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.273681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.273962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.274233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.283977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.287429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.289330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.304047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 317/636] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:32.304873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fe79be84-0d39-49b1-ab2e-d6b1d7f29eb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:32.306591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:32.307475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:32.308458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:32.308458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:32.308458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:32.308458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:32.308458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 320/637] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:32.309064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08266ff9-2c78-46e0-8e0d-842aabe42ec0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.309342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08266ff9-2c78-46e0-8e0d-842aabe42ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.309607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08266ff9-2c78-46e0-8e0d-842aabe42ec0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.310824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 321/638] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:32.311606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08266ff9-2c78-46e0-8e0d-842aabe42ec0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.412793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984fe2a-4894-4a91-8948-7b4176d1bfe2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.414887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08266ff9-2c78-46e0-8e0d-842aabe42ec0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.426423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.446296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.446942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.447220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.447332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.447490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.448015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.448290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:32.448378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.448566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.448672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.449340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:32.449649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.450008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6378f-bc3c-4312-b91a-33a2376e5000 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.451078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.461936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 318/639] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:32.467229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 322/640] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:32.467390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.467483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.467568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.467568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.481734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:32.482910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:32.517087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.547943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 319/641] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:32.548742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.562720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db87ba05-f1f3-4d20-92bc-6a87c8904e92 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.595457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.596137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.596417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.596692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.597396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:32.597698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.599043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.610453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 323/642] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:32.611077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.662604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7245d19-bafe-45d2-9d74-fa086aa0c2f6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.664177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.694434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.695209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.695490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.695804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.706734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:32.707839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4763e4-0a12-4e88-886f-4e80e5fa060a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:32.714350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:32.715389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 320/643] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:32.716258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.721320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.741536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.742178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.742456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.742732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.743558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=f310f6d84a3d42df9d473677ac5b6557) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:32.743897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.745321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.751330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 324/644] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/role_assignments?user.id=f310f6d84a3d42df9d473677ac5b6557&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:32.752714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.826047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b467e-7f62-4ea7-956b-504a2ac057f0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.863715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.864925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.865431 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.865695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/e168d14c0eac4ee2a5acdc23c39118d9/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.868474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37797fa-602c-427f-9f02-92d93f1ba1d5 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:32.883462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=e168d14c0eac4ee2a5acdc23c39118d9, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:32.884957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:32.903711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:32.904495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:32.904775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:32.905045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:32.919590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:32.925274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:32.926475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:32.927653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 325/645] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:32.928750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.951713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 321/646] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:32 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/e168d14c0eac4ee2a5acdc23c39118d9/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:32.951713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:32.973118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:32.976075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.069514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d08a7-be55-4591-ba89-de50eb925842 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.091321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f8c9a-0bdd-4a37-b027-eb6ec27f4b9b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.111156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.112004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.112279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.112556 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.121679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.123352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:33.139464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:33.140544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:33.141492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 326/647] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:33.142088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.142866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.143916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.144465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.144964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.145985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:33.146542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.148307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.160958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 322/648] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:33.162725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.260919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.280677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.285157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bc405-c873-4c1e-b433-91fe79391ca0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.290064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc63fd-f8a1-47d2-a742-5438dd8b9932 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.329081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.329996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.330422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.330692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.331034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.331491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.331944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.332266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:33.332377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.332815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.333295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:33.333670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.334756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.335224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.346692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 327/649] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:33.347811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.355146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 323/650] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:33.356207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.470866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784ffea-67e0-42f7-b276-8f3ac4dc8b73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.475189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed54f9-5975-45aa-9565-6ff0411458c9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.507962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.508652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.509022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.509392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.515105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.520958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.522177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.522599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.522809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.523222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.531955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.536112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.537750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:33.543060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:33.544207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:33.545289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:33.545289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:33.545289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:33.545289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:33.547443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 328/651] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:33.548352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:33.562189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:33.563309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:33.564417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:33.564417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:33.564417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:33.564417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:33.565402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.567796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 324/652] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:33.569034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.661298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b56f-1755-4f9f-904a-9a6684690802 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.698609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.699432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.699868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.700262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.700829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde6abde-a9b6-4dfa-8936-97ee03269ea5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.747313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.748736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.749194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.749608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.751017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:33.751680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.753478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.771429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 325/653] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:33.772727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.778725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.780905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.794465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.831957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:33.891014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ebfbc-ce3d-4659-84fa-7ca08df2dc69 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:33.898987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 329/654] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:33.900483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:33.934536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:33.936072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:33.936532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:33.936974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:33.937824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:33.938509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:33.943591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:33.955882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 326/655] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:33.956690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.021786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82817957-727d-48ea-953a-8880fb9c9997 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.037612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.063185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.064157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.064563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.064932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.066253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54652fd1-9553-4bfa-891d-b5e3f135cd56 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.070364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.076849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.078727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:34.098298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:34.099270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:34.100583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 330/656] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:34.101163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.114578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.115914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.116408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.116936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.129170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.131100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:34.150532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:34.151437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:34.152324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:34.152324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:34.152324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:34.152324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:34.154535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 327/657] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:34.156257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.229697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c6cc3-52a6-43ee-8820-676a6b173042 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.267827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.268811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.269112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.269386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.271119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c5295-b9ac-4a66-a64e-7227c88f7ff3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.282294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.282294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:34.294847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:34.295911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:34.296851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 331/658] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:34.297440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.311877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.312895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.313377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.313848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.314895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:34.315402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.317169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.329734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.336505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 328/659] 10.222.0.41 () {56 vars in 1139 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:34.337573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.353941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.406971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4690ac-f8d2-422d-a3e8-485370b5ce5f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.440501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aed2d-105f-4373-8711-6b5b45531a98 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.448775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.449610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.450019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.450391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.451584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:34.452019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.454608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.474726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 332/660] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:34.474726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.486561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.488149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.488847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.489276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.490490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:34.490970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.493368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.514029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 329/661] 10.222.0.41 () {56 vars in 1137 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:34.515552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.578560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.589672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9125c-3af2-4d4f-9c31-b055e242aa42 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.614477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.627043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.627968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.628562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.628990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.649550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2410735f-fabe-49a0-a31b-926375a9dfd3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.685727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.687268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.687788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.688162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.696023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.697432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:34.713050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:34.714238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:34.715286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:34.715920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.721532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 330/662] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:34.721532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.722366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.832641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.838060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90480ed6-95dc-47b0-a8f7-52741458ed38 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:34.849103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 333/663] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:34.849103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:34.882677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:34.888844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:34.889718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:34.890061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:34.890395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:34.970318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:34.971976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:34.984684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b198085-0a75-4cf8-b1f4-0fd67b4ce91e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.021229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.021886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.022165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.022438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.023211 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:35.023516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.024959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.042745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 334/664] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:35.043641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.096316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.096653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 331/665] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:35.097763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.130155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.149822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44df281-b27c-4436-bbc0-de5e259e4849 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.188122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.188913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.189192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.189466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.199820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.201245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.201556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e440-ea4d-4c37-a760-7a36cf439698 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:35.219623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:35.220791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:35.221822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:35.221822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:35.221822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:35.221822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:35.225222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 335/666] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:35.226028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.245222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.245877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.246161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.246434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.247252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=bb84e744681149aea78fcc984605bf33) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:35.247562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.249051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bdd263-64ce-426a-95e8-5a85c2e6db0a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.263024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 332/667] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/role_assignments?user.id=bb84e744681149aea78fcc984605bf33&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:35.263024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54e7a-ba03-4156-9bf6-b9ab19a044cd None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.263024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54e7a-ba03-4156-9bf6-b9ab19a044cd None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.263024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b54e7a-ba03-4156-9bf6-b9ab19a044cd None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.263024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 333/668] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:35.263622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54e7a-ba03-4156-9bf6-b9ab19a044cd None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.349998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dbf8d7-0a9c-4ade-af2e-b37bfbec8e23 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.351633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.376532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.384357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b54e7a-ba03-4156-9bf6-b9ab19a044cd None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.389708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.390737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.391017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.391286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.402597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.404054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-003dfe3f-38a7-4938-9612-5e733134b4ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:35.425151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:35.426112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:35.432375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:35.432375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:35.432375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:35.432375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:35.432599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.432700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 336/669] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:35.432781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.432781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.432781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.432966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.433043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:35.433114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.433191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.433260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.433327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9cb934-f319-4e8e-904e-6b3be4de3a38 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.445179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 334/670] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:35.446595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23cdbe-8bb5-4951-8d4a-c119c33278d2 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.446886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23cdbe-8bb5-4951-8d4a-c119c33278d2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.447161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23cdbe-8bb5-4951-8d4a-c119c33278d2 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.448616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 335/671] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:35.450189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23cdbe-8bb5-4951-8d4a-c119c33278d2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.457306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:35.458613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:35.537917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 337/672] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:35.539625 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.567885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23cdbe-8bb5-4951-8d4a-c119c33278d2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.610972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.611531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.612066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.612360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.613919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:35.614231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.615602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e91ff-51dc-43f7-a186-5de07df1b79e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.628533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.630719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 336/673] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:35.632124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.632414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.632694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.646756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c005aa-818c-4b09-81a1-b93fc2f74012 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.653985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:35.656185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:35.671599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.681072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.682127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.682621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.683268 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.690134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.691602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:35.706227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:35.707415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 338/674] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:35.708570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.738098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 337/675] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:35.739236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.822813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9463f33-5746-4eb1-8b47-516c3bf1591a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.854460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3cb2c-9491-4602-8da1-1a8b74070eb2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:35.858592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.859380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.859679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.859960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.870485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:35.878073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:35.879195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:35.880069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 339/676] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:35.880773 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.889471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.897189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:35.901160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:35.901160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:35.901160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:35.907295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:35.909144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:35.923105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:35.924041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:35.924912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:35.928981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 338/677] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:35.929805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:35.940544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:35.989162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86f9baa-3a09-499f-990d-f7a3fd6df38f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.025675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.026333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.026613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.026885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.027680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.028103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.029710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.040444 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d845b-9556-4fda-bf49-669eaffaf78b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.047999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 340/678] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.048941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.078582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.079380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.079680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.079975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.090742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:36.098520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:36.099453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:36.100601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 339/679] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:36.101274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.118349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.161367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf48dec-0f02-4a05-9834-ca53d00b9312 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.168994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.197061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.197726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.198010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.198286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.199023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.199362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.201681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.205594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e9aac-0f2e-4a24-87b3-fe8d2bc9998d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.214631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 341/680] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.214631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.248573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.249954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.250251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.250566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.251341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.251652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.253118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.265788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 340/681] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.266574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.324977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25923a-30dd-460c-900b-57d37a39e13c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.352009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.361876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.362632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.362937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.363220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.364050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=e527c9fc7ae1475595c311ac3215b51f) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.364361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.365021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea88c9-f81a-41a8-ab23-ab34d86d5310 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.365781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.383694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 342/682] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/role_assignments?user.id=e527c9fc7ae1475595c311ac3215b51f&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.383694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.406288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.421381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.421381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.421381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.421381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.421908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.421908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.421908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.431467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 341/683] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.432531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.511877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bf40d8-5d14-424b-8163-dbb01119d63f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.552629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.553874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.554184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.554753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.558446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89100a76-52d9-42b5-a42b-b89f2aa93db6 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.570525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.572909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:36.597419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:36.598513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:36.599592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:36.599592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:36.599592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:36.599592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:36.599884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.600297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.600297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.602348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.602465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 343/684] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:36.602552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=9110e7fab9084faa94b5d345bb9d519a) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.602552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.602712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.605523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.611463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 342/685] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/role_assignments?user.id=9110e7fab9084faa94b5d345bb9d519a&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.612925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.668584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.704667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.735370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a441d-219a-4f7a-90ea-d622b0672cd8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.774004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.775110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.775571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.775953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb2ae5c-97e6-430c-b6ef-5756f382caf9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.776287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.789589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.791344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:36.813340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:36.814323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:36.815188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:36.815188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:36.815188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:36.815188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:36.815188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 344/686] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:36.820394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.824196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.825990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.826446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.826874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.827892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.828394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.830066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.845215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 343/687] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:36.851142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:36.933442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:36.941693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e31c7e-447f-48ac-aa3d-ddc452ef6988 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:36.981081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:36.982150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:36.982636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:36.983162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:36.984250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:36.984762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:36.986353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:36.986553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ea45d-c521-48ea-b101-4dba3074b871 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.001699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 345/688] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:37.002712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:37.006842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:37.030214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.030583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.030795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.031915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.042539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.044056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:37.059241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:37.061077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:37.062890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 344/689] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:24:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:37.068868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:37.139099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609416f-9a55-4a7f-ad62-f953c0168b8e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.189958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9016e7-20ec-42cd-8a4f-cf8dda584b58 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.194170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.195136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.195522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.195935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.206195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.207828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.226630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.227551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.228043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.228457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.229423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:37.229868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.232210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:37.236304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:37.237438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:37.238428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:37.239304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 346/690] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:37.243895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:37.256252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 345/691] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:24:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:37.261283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:37.292045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:37.312389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:37.380921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbd589-8613-4a17-a3de-7f5f8a95eac9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.442640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da421463-e3e6-4df4-a7e9-af5d3e547397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.443581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421463-e3e6-4df4-a7e9-af5d3e547397 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.444034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421463-e3e6-4df4-a7e9-af5d3e547397 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.444459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da421463-e3e6-4df4-a7e9-af5d3e547397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.478188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b903e-ad9c-4d9a-bf13-fcaee1d3d051 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.537890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.538479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.538761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.539024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.539885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:37.540220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.541845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.551931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da421463-e3e6-4df4-a7e9-af5d3e547397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.553588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da421463-e3e6-4df4-a7e9-af5d3e547397 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.578191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 347/692] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 20:24:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:37.581687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:37.610719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:37.706613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:37.738474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 346/693] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:37.742629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787e757d-3910-4aed-ac5b-412404986b31 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.744471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc84804-99d2-40e6-977c-9ef1d3b6b7cb None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.744764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc84804-99d2-40e6-977c-9ef1d3b6b7cb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.745050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc84804-99d2-40e6-977c-9ef1d3b6b7cb None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.746648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 347/694] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:37.747380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc84804-99d2-40e6-977c-9ef1d3b6b7cb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:37.787061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.788044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.788499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.788922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.857062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc84804-99d2-40e6-977c-9ef1d3b6b7cb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:37.876674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.878749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.886460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:37.893693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:37.894433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.894731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.895086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.907555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:37.909700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b1a54402-d015-4bd9-a542-7ac81b31bc3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:37.928649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:37.929842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:37.930846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:37.930846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:37.930846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:37.930846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:37.934765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 348/695] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:37.936667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:37.936826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:37.937105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:37.955497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:37.956709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:37.968629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.007699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 348/696] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:38.008584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.051690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 349/697] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:38.052476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.139151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa6ba-1126-456c-a7fe-2c48d330dff1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.177892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bb115-113e-4bc2-b22b-0630a2d1466d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.184044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.184949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.185430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.186457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.187394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:38.187949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.190383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.192637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.212675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 349/698] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:38.212874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.221614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.222409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.222696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.222974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.234015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.239443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.254070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:38.258510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:38.259579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:38.260664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:38.260664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:38.260664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:38.260664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:38.267072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 350/699] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:38.267072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.353716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2047f1-491e-4ced-86df-adef737fe1d3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.381321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac193d5-7d45-4452-99a6-d3ee2adf1ad3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.391037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.392206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.392329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.392723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.405343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.406879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:38.424781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:38.425734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:38.426609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:38.426609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:38.426609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:38.426609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:38.426609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 350/700] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:38.427509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.430509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.432420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.432855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.433294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.442882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.444812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:38.464024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:38.465042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:38.465890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:38.471695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 351/701] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:38.471695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.499522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.538314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.555203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13371437-1261-4f8f-86a2-bed3acbdfde8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.592079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.592774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.593039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.593505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.594495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:38.594953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.596549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.596981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e1057-11ee-43c6-a849-eb9b6d93dda1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.610750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 351/702] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:38.613109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.645749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.646410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.646860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.647739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.648877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:38.649328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.651096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.670155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 352/703] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:38.672443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.722472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de8f855-542c-4059-b5df-a662f994f704 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.766333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.767399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.767948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.768543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.769566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:38.770145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.771853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.776773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.787267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 352/704] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:38.788594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.793117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:38.794690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882edc04-97b6-4cd2-875a-3406c9e54d92 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.832465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.833508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.833997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.834436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.844766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.846510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:38.862014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:38.862953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:38.863859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:38.866454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 353/705] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:38.867864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:38.902416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d404c50-7011-45b1-bc80-a8c53c1bfcff None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:38.940164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:38.940928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:38.941426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:38.941918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:38.942813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:38.944650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:38.978681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404ca95-23be-4dbd-bef6-cb5df4e38da3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.016127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:39.016767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.017049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.017326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.018211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=f310f6d84a3d42df9d473677ac5b6557) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:39.018525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:39.020061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c9f8f-5f8a-4ee4-85c1-6bd0f2330884 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:39.021489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 353/706] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:24:38 2022] POST /identity/v3/users => generated 254 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:39.023693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.026549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 354/707] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:38 2022] GET /identity/v3/role_assignments?user.id=f310f6d84a3d42df9d473677ac5b6557&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:39.102240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:39.151277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04abe46b-78c3-4352-ad67-1f98c588386d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.192206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:39.192892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.193177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.193455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.194259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:39.194608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:39.196105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:39.200782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d89f6-3a19-439f-9d6f-1f7c1dfba329 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.201083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d89f6-3a19-439f-9d6f-1f7c1dfba329 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.201340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951d89f6-3a19-439f-9d6f-1f7c1dfba329 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.203081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 355/708] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:39.216763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f65d6-196e-42f2-9c97-e8c1dde9464a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.217077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f65d6-196e-42f2-9c97-e8c1dde9464a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.217317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f65d6-196e-42f2-9c97-e8c1dde9464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.218959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 354/709] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:39.226987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.248777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-158f65d6-196e-42f2-9c97-e8c1dde9464a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:39.250924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-158f65d6-196e-42f2-9c97-e8c1dde9464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:39.254087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:39.412624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc49182c-a00c-48e1-90f0-48dab4bea017 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.419508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 356/710] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:39.420973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c8b3f-1254-4c74-ab38-6df5976f0b8f None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.421257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c8b3f-1254-4c74-ab38-6df5976f0b8f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.421538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c8b3f-1254-4c74-ab38-6df5976f0b8f None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.427809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 357/711] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:39.429015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.429303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.429582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.457463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:39.458568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:39.460082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:39.474714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:39.475371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.475650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.475935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.476950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=e168d14c0eac4ee2a5acdc23c39118d9) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:39.477659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:39.479458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:39.487646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 355/712] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/role_assignments?user.id=e168d14c0eac4ee2a5acdc23c39118d9&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:39.488560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.588778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 358/713] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:39.609505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4dba8-90c0-441c-a829-bdbd090531ac None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.611277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.653318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:39.654130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.654413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.654748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] PATH_INFO: `/v3/services/network` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.656047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:39.657917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:39.674286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:39.675339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:39.676666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 356/714] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:39.681609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.741015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:39.742374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:39.780512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d20371-fad0-479f-a730-c7d5913b4faf None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.795200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6089122-4814-406c-b7b8-bd9a9236291e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.822881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:39.823912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.824532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.824820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.825757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:39.828042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:39.846267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:39.847229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:39.848076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:39.848877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:39.848877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:39.848877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:39.849099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:39.850538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] RBAC: Adding query filter params (name=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:39.850538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:39.850861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 359/715] 10.222.0.41 () {56 vars in 1129 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:39.852015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:39.856818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.857704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 357/716] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:39.862997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:39.969534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff3670-f281-46d7-9aa2-938d8ea3f4bb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.975198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed0f81-37e2-488d-8825-73131d2d21df None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:39.989555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:40.002959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:40.008387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:40.009019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:40.009397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:40.009824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:40.010613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:40.010922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:40.012339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:40.013689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:40.014513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:40.014798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:40.015076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:40.015805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] RBAC: Adding query filter params (type=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:40.016111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:40.017490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee5e99-fd3b-4ce2-9238-9313583f8753 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:40.017922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 360/717] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:40.023261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:40.027776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 358/718] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:24:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:40.141109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff79f52-a7af-42fe-b42e-fa916a6dc0f1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:40.177721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:40.178453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:40.178732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:40.179057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:40.179798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:40.180109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:40.181579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f4af2b-1c51-4c2e-94ad-16df24d107ad None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:40.188457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 361/719] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 20:24:40 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:41.688334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292676f-d095-4a9a-a7b9-edbdae4f0f7f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:41.688842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292676f-d095-4a9a-a7b9-edbdae4f0f7f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:41.688842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292676f-d095-4a9a-a7b9-edbdae4f0f7f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:41.690406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 359/720] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:41.695810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1a817-4260-4e32-8942-49957be52b78 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:41.696208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1a817-4260-4e32-8942-49957be52b78 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:41.696378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1a817-4260-4e32-8942-49957be52b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:41.720252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-eca1a817-4260-4e32-8942-49957be52b78 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:41.721450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-eca1a817-4260-4e32-8942-49957be52b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:41.798309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:41.809733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 362/721] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 06 20:24:41.838326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96abf8d-d7f7-40f8-8af3-246524945d54 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:41.838685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96abf8d-d7f7-40f8-8af3-246524945d54 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:41.838890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96abf8d-d7f7-40f8-8af3-246524945d54 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:41.840540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 360/722] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:41.847217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96abf8d-d7f7-40f8-8af3-246524945d54 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:41.847695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:41.848039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:41.848389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:41.886031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:41.887218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:41.976293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96abf8d-d7f7-40f8-8af3-246524945d54 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.010586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.011491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.011802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.012080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.013963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 363/723] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:42.023229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:42.030885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:42.031916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:42.032984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 361/724] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:42.040186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.047683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.103494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:42.209634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3fc794-504f-4ed5-99e9-4947b9634e04 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.230643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64f3f10-229e-4378-9581-804e1c73fa79 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.230771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:42.253698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.254429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.254722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.255002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.255776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:42.256087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:42.257470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:42.276143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 364/725] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:42.279683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.305579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.306478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.306937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.307266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.325948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:42.334518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:42.335529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:42.336400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:42.339138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 362/726] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:42.345504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.481921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b1a47-cf55-407e-9045-84494f4872fa None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.536981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.538161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.538645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.539105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.555688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:42.556192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5df32-1bfa-467b-bad3-156b15cc4259 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.568079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:42.569551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:42.590032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:42.590973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:42.591925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:42.591925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:42.591925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:42.591925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:42.592143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 365/727] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:42.597042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.638108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.638779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.639058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.639351 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.640132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:42.640435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:42.641828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:42.661425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 363/728] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:42.667246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.675112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:42.750238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ac7d0-78ad-4bb7-bb8d-0e1eb13e48a0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.790025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.790984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.791409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.791922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.792916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:42.793225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:42.794598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f92ef-2aac-4bdd-b4fc-587f159b104c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:42.811147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660a706-801f-43b0-94ca-6b8f4c1882b2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.813203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 366/729] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:42.814511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eccc0-d78f-4a53-bddb-b87cd5781f5a None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.814798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eccc0-d78f-4a53-bddb-b87cd5781f5a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.815074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eccc0-d78f-4a53-bddb-b87cd5781f5a None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.816634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 367/730] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:42.817726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f84018-b2f7-42f1-9aa1-9337b41c19f5 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.817990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f84018-b2f7-42f1-9aa1-9337b41c19f5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.818267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f84018-b2f7-42f1-9aa1-9337b41c19f5 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.819502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 368/731] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:42.820197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f84018-b2f7-42f1-9aa1-9337b41c19f5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:42.846791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.847683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.847874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.848143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.859968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:42.861421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ecdd27da-7676-4b65-8591-8b82148af686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:42.879639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:42.880580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:42.881461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:42.881461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:42.881461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:42.881461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:42.881461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 364/732] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:42.882244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b8b13-e52f-49d0-abbb-87ed20a217df None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.882356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b8b13-e52f-49d0-abbb-87ed20a217df None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.882616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b8b13-e52f-49d0-abbb-87ed20a217df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.899233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c9b8b13-e52f-49d0-abbb-87ed20a217df None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:42.900453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c9b8b13-e52f-49d0-abbb-87ed20a217df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:42.903564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:42.926182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:42.935008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f84018-b2f7-42f1-9aa1-9337b41c19f5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:42.971944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:42.972754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.973039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.973314 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.975790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 365/733] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:42.977549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:42.978549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:42.979029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:42.982745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:42.984798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:42.998224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:42.999427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:43.000925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:43.001842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:24:43.002769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:43.003811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 369/734] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:24:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:43.007802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.071235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 366/735] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:43.072317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.119447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76945fc-ebb3-4440-99bd-52a73e6ad23a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.142133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.152829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.153487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.153771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.154045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.154752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:43.155613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.157009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.173017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 370/736] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:43.173017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.173346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50fd24-4528-4f7d-97b8-731e013ea5cb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.185546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.205761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.206544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.206829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.221991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.221991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:43.225412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:43.226326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:43.227155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 367/737] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:43.227874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.274768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65652734-32bb-4291-9083-eef88ee2339a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.308093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.308763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.309042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.309318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.310134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:43.310442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.311890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a9906-24a3-4e61-8650-88fe43104219 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.325569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d98c5-27bd-4f88-9828-b386e9fff40b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.328793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 371/738] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:43.330014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb37ed-08f3-4afa-ad1c-4ca07da09132 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.330302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb37ed-08f3-4afa-ad1c-4ca07da09132 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.330580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb37ed-08f3-4afa-ad1c-4ca07da09132 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.331815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 372/739] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:43.332500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb37ed-08f3-4afa-ad1c-4ca07da09132 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.359785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.365191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.365840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.366118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.366391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.367099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.368603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.403791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.432696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb37ed-08f3-4afa-ad1c-4ca07da09132 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.464527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.465305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.465598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.465875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.470846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 368/740] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:24:43 2022] POST /identity/v3/services => generated 240 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:43.477315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.482768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.484856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:43.502937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:43.504149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 373/741] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:43.505091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.605241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e1d5b1-0aba-4347-9488-d40dd8d7bb24 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.620760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68faa0f-ef7c-4f1f-8816-b54f09c81358 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.643541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.644251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.644537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.644814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.645572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:43.645904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.647304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa6d62-70d9-4d72-9a19-eecd0fe7bbc9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.653759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.656866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 369/742] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:43.659136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.659136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.659136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.666034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.666694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.666974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.667246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.678632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:43.679848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:43.728194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.743333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.744835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.763693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 370/743] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:43 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 06 20:24:43.763693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.852098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 374/744] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 06 20:24:43.852098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.873661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cb2b2-8378-45ae-8a87-96dd2f06f109 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.882976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.909017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.911008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.911008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.911008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:43.911008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:43.911446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:43.912758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:43.934828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 371/745] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:43.935475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:43.953934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef33200-c81e-4433-8a5d-3a3d6a57a558 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:43.975264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:43.992540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:43.993328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:43.993612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:43.994190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.004890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.006333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:44.023827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:44.024777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:44.025676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:44.025676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:44.025676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:44.025676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:44.025676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 375/746] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:44.025945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.048979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5204c-727e-45af-8d7a-c864d2582330 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.085887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.086550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.086830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.087107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.087854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.089230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.111290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:44.126861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f1f13-515c-4c97-8766-7445c0439371 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.147690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 372/747] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:24:43 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:44.147690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.189631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.190437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.191219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.191510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.208627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.210206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:44.233989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:44.234965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:44.235909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:44.235909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:44.235909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:44.235909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:44.242918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 376/748] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:44.243998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.268803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:44.276146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe18a0a-b330-44ad-8012-8063ed2437a2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.331521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.334349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.334726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.335076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.351115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.353529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.374618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc243e5-1162-41b8-b196-0f231ddbc5d6 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:44.377303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:44.378157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:44.379055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:44.379055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:44.379055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:44.379055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:44.379055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 373/749] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:44.383171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.414373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:44.424568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.425801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.439153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.439153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.439153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:44.439153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.439153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.444896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 377/750] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:44.445895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.508345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cf0c4-0b21-4827-9a1b-604ce2502f95 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.544246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.545277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.546552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.546979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.547873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:44.548327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.550293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.562257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35c1f9-2c66-4b37-bef5-a692db69a6b5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.566650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 374/751] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:44.567648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.579853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:44.604461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.605555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.605708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.606061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.606913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:44.607412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.609578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.629021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 378/752] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:44.630065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.688873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd310c0-3269-4929-980a-a556e61ec030 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.706349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:44.728385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.729323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.729707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.730104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] PATH_INFO: `/v3/users/swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.738357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.740293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:44.755358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:44.756566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:44.757734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:44.759643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 375/753] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:44.760618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.771705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5667ce6-e9bf-47bb-9c25-be937034fbc7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.820571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.821886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.822733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.823171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.831284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:44.833937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.836172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:44.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:44.854758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:44.855882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:44.858693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 379/754] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:44.859623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.874752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb7371-0dd5-4f48-9a14-4e19a993f3ef None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:44.918915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:44.920280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:44.920743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:44.921420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:44.930714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:44.933300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:44.952709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:44.953909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:44.955009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:44.957632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 376/755] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:44.958574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:44.986200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a7d8c-9f3f-4506-b84d-975e71898701 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.001979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:45.023041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.023897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.024285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.024791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.025756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:45.026211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.027812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.047888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 380/756] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:45.049342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.095154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d8ddd-3566-4787-834c-a9791aac4d10 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.127589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:45.136078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.140495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.140495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.140495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.140495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:45.140495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.140872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.159078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 377/757] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:45.160056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.166685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8cad81-3f3a-472a-9314-7e6e3d945f06 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.214243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.214819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.215611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.215935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.216865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:45.217315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.219459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.241664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 381/758] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:45.242742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.278557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8fab77-984b-4724-b584-2367d4f3a63f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.285702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:45.315864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.316627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.316975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.317407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.384193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25414a3e-ffa3-4ac9-b64f-0e8353f0e61c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.387884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:45.401124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.402670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.434054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.435049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.435803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.437324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.438805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=9196a10f723649839a7eca20e99d84f4) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:45.439287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.441383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.450667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 382/759] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:45 2022] GET /identity/v3/role_assignments?user.id=9196a10f723649839a7eca20e99d84f4&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:45.451824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.570904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:45.579329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17a0096-4acd-4e2c-ab67-65ab8ba705f2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.595215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 378/760] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:24:45.600428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.619563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1e8b2-a80f-4a32-99b6-73bc736c507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.620747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e8b2-a80f-4a32-99b6-73bc736c507a None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.621184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e8b2-a80f-4a32-99b6-73bc736c507a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.621611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1e8b2-a80f-4a32-99b6-73bc736c507a None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/f310f6d84a3d42df9d473677ac5b6557/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.626213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1e8b2-a80f-4a32-99b6-73bc736c507a None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=f310f6d84a3d42df9d473677ac5b6557, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.627874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1e8b2-a80f-4a32-99b6-73bc736c507a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.710349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 383/761] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:45 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/f310f6d84a3d42df9d473677ac5b6557/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:45.723046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:45.738954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0cb6e-db68-4866-b846-037318f79912 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:45.784877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:45.785701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.785983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.786257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.799108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:45.800689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:45.805312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c52e9-104d-42e6-9d5c-9a4ce701f10d None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.805668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c52e9-104d-42e6-9d5c-9a4ce701f10d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.805896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c52e9-104d-42e6-9d5c-9a4ce701f10d None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.807273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 384/762] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:45.814989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:45.815602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:45.816112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:45.820808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:45.821841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:45.823180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:45.823180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:45.823180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:45.823180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:45.823180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 379/763] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:45.828566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.840982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:45.843653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:45.932208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 385/764] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:24:45.953906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:45.991152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28abfa3-1406-40b6-916e-9d03d653ecaf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.029348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.030494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.031094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.031571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.032516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:46.033037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.034994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.044358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.047401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.049398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 380/765] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:46.054241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.067160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9adbb-52c7-405c-b526-213ec32d72fe None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.124847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.126043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.126506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.126988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] PATH_INFO: `/v3/services/network` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.127847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.129406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:46.141972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:46.142990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:46.143863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:46.152105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 386/766] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:45 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:46.153508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.249856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c420f1-514f-440c-a97f-729f2498e441 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.263892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67944be0-c545-46d6-9e60-e84928feb198 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.303468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.304438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.304867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.305296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.306144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] RBAC: Adding query filter params (name=network) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:46.306659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.308336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.319146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 387/767] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:46.320540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.322770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.323574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.323884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.324161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.337518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.348696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.350088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:46.384704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:46.385818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:46.387070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 381/768] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:46.391385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.439173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9fc8a-d6d3-4fed-b9cb-5d1eb70fec6f None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.472034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.478683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.479627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.494530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.494530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.494530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:46.494530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.494530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.494530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 388/769] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:46.496561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.572229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fbee0-7ca4-45ba-b18b-cc5759eae5b2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.583421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.612080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0aaa76-ca84-4e8e-8535-3e3ce01137c4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.630011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.630668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.630946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.631219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.632047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:46.632349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.634236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d14fe0a-fb4c-45fd-8ec3-0c574d32c740 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.648238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.649171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.649651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.650186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.651179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] RBAC: Adding query filter params (interface=public, service_id=afaff8c6380149fdb74284dcb78f73c6, region_id=RegionOne) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:46.651720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.653482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe59da-87b9-482e-806f-9d584cff77ab None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.653790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 382/770] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:46.655715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9feff-001a-45de-878e-db6d0ed89e9e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.656017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9feff-001a-45de-878e-db6d0ed89e9e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.656280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9feff-001a-45de-878e-db6d0ed89e9e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.657985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 383/771] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:46.658957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9feff-001a-45de-878e-db6d0ed89e9e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.671815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 389/772] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/endpoints?service_id=afaff8c6380149fdb74284dcb78f73c6&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:46.673575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.674063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.674546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.696649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:46.698643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:46.779323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9feff-001a-45de-878e-db6d0ed89e9e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.784086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 390/773] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:46.784957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.804690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.816790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.824582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.824582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.824582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.824582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=e168d14c0eac4ee2a5acdc23c39118d9) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:46.824582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.824582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.835836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 384/774] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/role_assignments?user.id=e168d14c0eac4ee2a5acdc23c39118d9&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:46.837386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:46.876090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:46.908834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8c676-ed9b-484b-9050-9476b9a92c8a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:46.945942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:46.946757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:46.946923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:46.947204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:46.948021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:46.949490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:46.956017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 391/775] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:47.040209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca52ce-c5d5-4146-a52c-1e2164443807 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:47.092046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:47.093463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:47.093751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:47.094027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:47.094705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:47.100501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:47.113877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:47.114875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:47.116195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 385/776] 10.222.0.41 () {56 vars in 1129 bytes} [Mon Jun 6 20:24:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:47.121621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:47.250511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7fa8e-2a86-4fc3-a94b-bfde670e62a8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:47.292141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:47.292810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:47.293100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:47.293376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:47.294084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:47.294396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:47.295823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:47.296886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:47.301745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 392/777] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 20:24:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:47.307071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:47.361611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:47.419183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb903494-3f3f-43ee-b4ee-c6d68e482a22 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:47.463133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:47.463824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:47.464114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:47.464392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:47.465100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:47.465408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:47.466778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:47.476006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 386/778] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 20:24:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:47.481440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:47.611063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e1c620-edc4-43dc-b27d-2d0c2bbe1d07 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:47.645900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:47.646612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:47.646902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:47.647169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:47.647998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] RBAC: Adding query filter params (interface=public, service_id=03bfb73852fa4705b1f5765c559b387c, region_id=RegionOne) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:47.648315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:47.649747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e6c89-c03a-40f2-bc46-00575215ffef None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:47.655512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 393/779] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:24:47 2022] GET /identity/v3/endpoints?service_id=03bfb73852fa4705b1f5765c559b387c&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:47.660724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:47.787433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b218e78-9efb-4119-a725-6e13fb84b12b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:47.801164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:47.823216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb3475-ad79-49cc-b451-e9d198092b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:47.823914 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb3475-ad79-49cc-b451-e9d198092b59 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:47.824199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb3475-ad79-49cc-b451-e9d198092b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:47.824478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb3475-ad79-49cc-b451-e9d198092b59 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:47.825166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb3475-ad79-49cc-b451-e9d198092b59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:47.826554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb3475-ad79-49cc-b451-e9d198092b59 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:47.832309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 387/780] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:24:47 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:48.257044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ada3d-c1f6-4ac7-96fe-9607f899a034 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.257592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ada3d-c1f6-4ac7-96fe-9607f899a034 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.257592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ada3d-c1f6-4ac7-96fe-9607f899a034 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.259174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 394/781] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:48.264730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ee111-3762-4988-99d9-466cf92a8ab6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.265168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ee111-3762-4988-99d9-466cf92a8ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.265360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ee111-3762-4988-99d9-466cf92a8ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.286325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-020ee111-3762-4988-99d9-466cf92a8ab6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:48.287706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-020ee111-3762-4988-99d9-466cf92a8ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:48.364309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 388/782] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:48.388550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ada3d-c1f6-4ac7-96fe-9607f899a034 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:48.413562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a4cb-dc13-4570-bccf-4ef43616101e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.414179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a4cb-dc13-4570-bccf-4ef43616101e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.414657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a4cb-dc13-4570-bccf-4ef43616101e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.416542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 389/783] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:24:48.422379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.423002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.423480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.441027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:48.442553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:48.488140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:48.552019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 390/784] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:24:48.562806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ada3d-c1f6-4ac7-96fe-9607f899a034 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:48.611498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:48.612735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.613240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.613749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.617442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:48.631979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:48.641474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:48.642390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:48.643203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:48.645920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 395/785] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:48.651877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:48.708056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:48.762071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e888a5-31e5-4650-a3b6-9085973bfbce None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:48.765026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f91b5-b7cf-4d71-bef7-483ab378aadb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:48.785964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:48.803040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:48.804318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.804803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.805275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.817895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:48.826086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:48.827632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:48.828491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:48.835921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 391/786] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:48.837381 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:48.846552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:48.847232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:48.847516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:48.847818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:48.848568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:48.848871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:48.851767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:48.885982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 396/787] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:48.892631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:48.950558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:48.958355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db31e2a-ee30-4d12-8068-94d473b6f507 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.002362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.003421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.003935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.004407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.006121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:49.007563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.010075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.038662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 392/788] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:49.056699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.081048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37435f8a-7507-43f7-8c86-c07f7e2562d7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.118926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.120168 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.120653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.121185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.143311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.151984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:49.199686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:49.200622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:49.201492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:49.201492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:49.201492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:49.201492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:49.208102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 397/789] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:49.213199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.223214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182bec-8b48-458f-836d-bee5bcf2e6a0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.272276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.273901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.276086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.276380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.316226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:49.316733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.318109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:49.353523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:49.354975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:49.360733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:49.360733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:49.360733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:49.360733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:49.360733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 393/790] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:49.372718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.380723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6a141-f675-4133-8594-96e754c83ff4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.401091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:49.458239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.459317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.459831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.460306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.461238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:49.461789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.465409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.502017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 398/791] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:49.507391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.538888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6fdf5d-c37d-4d97-98aa-3a143e35dfe3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.577873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.578892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.579373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.579875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.580815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:49.581316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.582890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.600783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 394/792] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:49.603128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.686476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:49.723926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45b6b9-3b7e-4f3a-b49d-3a6fc7fc1c8f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.743080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4961648e-56ce-4b29-8d6a-9ca1581733f8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.764220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:49.767602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.768928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.769562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.770352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.780719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.782057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.782782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.782891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.783132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.783677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.791415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.797479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9f64706b-4fd5-49b1-a5f1-2976e232f9f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:49.800749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:49.801653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:24:49.802611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:49.806616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 395/793] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:49.808002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88691fa-b778-48d5-affa-93a3788d1e1a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.808288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88691fa-b778-48d5-affa-93a3788d1e1a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.808564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88691fa-b778-48d5-affa-93a3788d1e1a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.810105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 396/794] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:49.810805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88691fa-b778-48d5-affa-93a3788d1e1a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-14e7b678-89c9-40b0-a1a4-449378543a22 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:49.818489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:49.819417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:24:49.820427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:49.824097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 399/795] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:49.825820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.826312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.826943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.845446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:49.847067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:49.919077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88691fa-b778-48d5-affa-93a3788d1e1a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:49.920606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 400/796] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:49.921717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:49.972410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:49.979519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:49.980135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:49.980421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:49.982374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:49.982374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:49.982374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:49.983270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.001339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 397/797] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:50.002170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.015617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.032957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd532ec-e97c-4b39-8507-53bfdb3ef9a5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.068645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.069312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.069591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.069864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.070679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:50.070987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.072452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45b50d-41b4-43a1-85dc-014e60e64dbd None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.091028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 401/798] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 20:24:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:50.092934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b41ee5-9db0-43cf-86af-0a9cded6935a None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.093225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b41ee5-9db0-43cf-86af-0a9cded6935a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.093500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b41ee5-9db0-43cf-86af-0a9cded6935a None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.098728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 402/799] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:50.098728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b41ee5-9db0-43cf-86af-0a9cded6935a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.112849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c819b70-4fb6-4126-b9fd-e4894f7c05fc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.152899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.153793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.154068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.154334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.165932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:50.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:50.174821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 398/800] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:50.180536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.208353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b41ee5-9db0-43cf-86af-0a9cded6935a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.241800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.247433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.248126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.248409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.249223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.266778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.300762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb6fe5-44ae-4e76-bda4-3ac830f5616a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.324457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.326087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.337327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.342852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.342852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.342852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.350492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.351917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-82e28a9d-2940-4afa-996f-2d7c9c764efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:50.369307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:50.370201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:50.371079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:50.371079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:50.371079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:50.371079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:50.371079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 399/801] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:50.372475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.372475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.372475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.390303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:50.391494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:50.434891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 403/802] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:24:50.435544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.471686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 400/803] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:50.471686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.493505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.500526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.548681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94af6f91-70c9-499d-97fc-3d86c161b326 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.578234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855e2e7-7666-4dbb-bb80-f54e17b7b2ac None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.584202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.584866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.585145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.585803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.586550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:50.586852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.588591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.601892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 404/804] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:50.601892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.618688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.619452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.619837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.620115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.620861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:50.621163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.622526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.634107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 401/805] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:50.634875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.696676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e832ae5-7e9e-4363-a8ee-9489d78d6ca4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.703629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.722459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.728605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.729427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.729730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.729974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.731422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53a7b2b-e345-48bf-b11c-c39dc4e9a191 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.741995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.743481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:50.760395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:50.761378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:50.762345 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:50.762345 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:50.762345 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:50.762345 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:50.762345 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 405/806] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:50.762681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.767041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.767855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.768137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.768408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] PATH_INFO: `/v3/services/network` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.769053 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.770408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-29fbbdf5-6620-4db3-b709-b651a3e2f12b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:50.782213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:50.783596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:50.784515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 402/807] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/services/network => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:50.785284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913cec0-b2db-453d-a12e-eeee7c806a4a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.785284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913cec0-b2db-453d-a12e-eeee7c806a4a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.785425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913cec0-b2db-453d-a12e-eeee7c806a4a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.786562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 403/808] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:50.787242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913cec0-b2db-453d-a12e-eeee7c806a4a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.863929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1cd11-9c9b-4299-b0f1-e4e89c2b1e87 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.885643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913cec0-b2db-453d-a12e-eeee7c806a4a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:50.896273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.897047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.897327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.897599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.907844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.909533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.918147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:50.918918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:50.919204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:50.919482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:50.926473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:50.927437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:50.928703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:50.928703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:50.928703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:50.928703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:50.928935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:50.929016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 406/809] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:50.929086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:50.929154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.941713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:50.948127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:50.949163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 404/810] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:50.950218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:50.965571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.034071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc177f-893d-425e-a671-1dc54a048841 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.048006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011110c9-d82e-4f5f-83a3-63d090eb1371 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.066872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.067526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.067840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.068116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.068828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.069133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.070524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f738c-ce15-4af9-a336-7f3e559c1b99 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.081039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.081911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.082017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 407/811] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.082091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.082222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.082893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.082996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] RBAC: Adding query filter params (name=network) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.083181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.083265 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.083458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.084564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.089855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 405/812] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:24:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.090620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.100530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:51.101735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:51.164180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.167054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.168530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 408/813] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:51.169320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.200639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18bccb-e96f-41ae-bb1d-f1831c514740 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.234964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.235628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.235943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.236228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.236946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.237252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.238626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.254916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 406/814] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.254916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.272075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36100428-bcaf-46ac-997e-b01cda29f9ed None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.306291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.306938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.307255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.307526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.308335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.308639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.310066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.321879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 409/815] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.322645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.351152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf49c9-f2fd-46a7-8bc9-08b449a45e73 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.381589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.384627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.385402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.385689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.385958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.392465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.393895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.395584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:51.408237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:51.409554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 407/816] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:51.410465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.427416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68998837-dccf-4034-8b8e-5f7bc3161d67 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.460614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.461276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.461559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.461836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.462528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] RBAC: Adding query filter params (type=network) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.462834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.464247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.473498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 410/817] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.473498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.509256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad8f4b-8960-4d4d-8d05-fe850cf3cbb2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.542656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.543455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.543765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.544042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.544712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.546602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:51.558437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:51.559360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:51.560457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 408/818] 10.222.0.41 () {56 vars in 1129 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:51.561865 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.569867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06e43c-25c7-4735-a60b-8378129c1041 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.591827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.602438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.604668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.604668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.604668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.611687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.613120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.619582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:51.627681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:51.628665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:51.629579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 411/819] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:51.630152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.660908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b0a1f8-26f1-4015-a92f-a8f0af2790fb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.695216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.696057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.696333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.696603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/9196a10f723649839a7eca20e99d84f4/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.710560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=9196a10f723649839a7eca20e99d84f4, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.712000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.726672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf4e5f-a324-4394-9839-39ced46c6df0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.758584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.759262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.759543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.759847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.760611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.760917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.762311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.773806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 412/820] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.774613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.786189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 409/821] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:51 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/9196a10f723649839a7eca20e99d84f4/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:51.787744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:51.825608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.885322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dae375-0fdb-4a73-b44c-b8609ae27f23 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.902791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab04c9-44b1-4830-9f12-408396f6f9fb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:51.917484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:51.925155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.925993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.926275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.926547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.927296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.928736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.941014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:51.941837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:51.942182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:51.942522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:51.943294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:51.943719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:51.945184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:51.952949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 410/822] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:51.952949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.030875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 413/823] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:24:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:52.032350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.061984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0943a8-945c-410b-8f0d-db07177722a7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.124442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.125146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.125429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.125706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.126563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:52.126872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.129028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.152765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:52.160038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 411/824] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 20:24:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:52.161398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83a95f1-6e75-42b3-9220-87e7faf3e450 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.167042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.198316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:52.201191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.202680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.203125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.203437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.204445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=f310f6d84a3d42df9d473677ac5b6557) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:52.204858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.206878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.213882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 414/825] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:52 2022] GET /identity/v3/role_assignments?user.id=f310f6d84a3d42df9d473677ac5b6557&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:52.215015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.290093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42da03-0bac-481d-9c95-c12d5e4334c6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.331479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.332516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.332977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.333407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.334274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:52.334733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.336530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.343265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba95cdf-699c-43b1-b4f4-02ff25b46d76 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.343508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 412/826] 10.222.0.41 () {56 vars in 1124 bytes} [Mon Jun 6 20:24:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:52.348919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.380335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.381284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.381665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.382210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.395550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.397316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:52.418181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:52.419315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:52.420483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:52.420483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:52.420483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:52.420483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:52.426723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 415/827] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:52.427966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.448956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:52.483908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:52.494941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b81014-6c08-4f54-ae0b-8a8836ef8673 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.541247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.542154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.542435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.542719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.543439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.544873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.558025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43135dd1-9ebf-4631-8ea4-5878e80250af None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.594847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.595511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.595830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.596109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.596844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:52.597140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.598524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.604806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 413/828] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:24:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:52.612158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 416/829] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:52.617971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.741110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c4a32-8cb7-4bb4-a78e-d34a2c8f40b1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.779202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.780565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.781050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.781522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.789699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.791524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:52.807289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:52.808249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:52.809230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:52.812364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 414/830] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:52.818268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:52.906741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:52.930195 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52daccd4-f7a1-46ad-91b2-4ae990cdccde None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:52.969561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:52.970221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:52.970553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:52.970832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:52.971599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:52.971942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:52.973365 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb986d-7bbd-4648-80e7-d2e2886f185d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:52.985963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 417/831] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:52.991626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:53.001478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:53.097488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e200-3816-417c-863f-a68a0ce9f8bd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:53.133830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a8204-f7f8-4999-8d4c-559fb5590ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:53.134717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8204-f7f8-4999-8d4c-559fb5590ca6 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:53.134999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8204-f7f8-4999-8d4c-559fb5590ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:53.135275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8204-f7f8-4999-8d4c-559fb5590ca6 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/e168d14c0eac4ee2a5acdc23c39118d9/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:53.137295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a8204-f7f8-4999-8d4c-559fb5590ca6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=e168d14c0eac4ee2a5acdc23c39118d9, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:53.138740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a8204-f7f8-4999-8d4c-559fb5590ca6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:53.212646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 415/832] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:24:52 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/e168d14c0eac4ee2a5acdc23c39118d9/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:24:54.161116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784d8f3-b827-46dd-a52f-35900884bede None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.161641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784d8f3-b827-46dd-a52f-35900884bede None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.161641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2784d8f3-b827-46dd-a52f-35900884bede None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.163422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 418/833] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:54.168961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.169364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.169532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.187399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:54.188614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:54.256158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:54.262072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 416/834] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:54.286441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784d8f3-b827-46dd-a52f-35900884bede None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:54.402188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2784d8f3-b827-46dd-a52f-35900884bede None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:54.437535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:54.438247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.438524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.438798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.441660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:54.450278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:54.458063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:54.458964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:24:54.459824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:54.462694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 419/835] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:24:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:54.465484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:54.567178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c75c6c-af09-4f91-85c7-44a58e9912f5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:54.600975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:54.601635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.601911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.602180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.602922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:54.603282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:54.604761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b540ce40-cdbf-433d-ba55-f1e2a42eb161 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:54.619531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 417/836] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:24:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:54.621016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:54.675036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca21ef-77d9-4440-977f-1a2190af2827 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.675359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca21ef-77d9-4440-977f-1a2190af2827 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.675597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca21ef-77d9-4440-977f-1a2190af2827 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.677244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 418/837] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:24:54.682022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.682303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.682580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.699760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:54.701003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:54.734129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d45899-4232-4cb7-8050-476c1e666fa1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:54.772769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:54.773561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.773895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.774177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.775891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:54.786797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:54.788862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:54.799797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 419/838] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:54.811820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:54.812845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:54.813846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:54.813846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:54.813846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:54.813846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:54.815873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 420/839] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:54.824125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:54.825357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:54.879584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:54.932467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226416c-6cd1-4e98-8045-d5b41bf82c6f None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:54.940438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16f73fb-a155-4ecb-affe-e47adaff894b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:54.968970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:54.969717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.970220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.970328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.970911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:54.972333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:54.978477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:54.979133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.979414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.979721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.980436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:54.980738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:54.982274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-e0b45391-6c6f-419e-a8aa-d0fb73fdd055 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:54.985511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:54.986639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:54.987480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 421/840] 10.222.0.41 () {56 vars in 1132 bytes} [Mon Jun 6 20:24:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:54.988770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb925-d549-449a-ad9b-898d90147680 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.988770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb925-d549-449a-ad9b-898d90147680 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.988770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbb925-d549-449a-ad9b-898d90147680 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:54.989868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 422/841] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:54.994974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:54.995267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:54.995546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.002538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 420/842] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:55.003453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.017161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:55.018376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:55.064240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.101641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 423/843] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:24:55.102824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.128824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10669f3e-5773-4f39-86d6-553c8e9dee52 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.154565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.173525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.174196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.174487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.174769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.175481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:55.175826 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.177204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.182056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 421/844] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:55.182843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.229883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae91a5-afae-48ab-a877-6c2687b4edaf None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.268578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.269405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.269691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.269973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.277518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.278973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:55.294940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:55.296609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:24:55.297772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:55.301514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 424/845] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:55.302399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.317009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d6112-03b4-4e40-9d4d-fd765be3ce09 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.332972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.363233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.364365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.364799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.365223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.366050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.367619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:55.380493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:55.381728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:24:55.382733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:55.384943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 422/846] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:55.385874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.421364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a83e3-8c18-4b9c-b61d-a57d862c91b8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.423039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.458512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.459337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.459989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.460279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.461140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:55.461456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.462940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.468191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 425/847] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:55.468993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.531209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd7339d-edeb-4a25-9ccc-6e282840be35 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.570666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.571349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.571636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.571941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.572800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:55.573229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.574619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12221631-1e85-45b0-9ee3-49800e9030a3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.588924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1e833-c0d2-4926-9de3-4800cdb41c68 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.595543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 423/848] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:55.597118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eebb8-df1e-41ac-bbd9-b5c6dee0bc42 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.597409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eebb8-df1e-41ac-bbd9-b5c6dee0bc42 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.597688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586eebb8-df1e-41ac-bbd9-b5c6dee0bc42 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.599176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 424/849] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:55.601918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eebb8-df1e-41ac-bbd9-b5c6dee0bc42 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.638224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.639765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.640235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.640765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.641785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.641921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:55.642157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.644547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81edddc6-d992-41b4-a266-7fbb0e512094 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.650078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 426/850] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:55.651397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.651691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.651980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.672941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:55.674225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:55.705988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.720653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586eebb8-df1e-41ac-bbd9-b5c6dee0bc42 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.755340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 427/851] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:55.756224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.764047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.765311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.765624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.765899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.836284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.837626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.870122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac37b0-bddd-4aa3-8ca3-e4f8d4a6ba69 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:55.873239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.907026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:55.907718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:55.908005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:55.908576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:55.909248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:55.909559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:55.910964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:55.915722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 428/852] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:55.916504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:55.945275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:55.966722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 425/853] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:24:55.975690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.038591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bca92e-e5b6-4975-a267-502674c8a19b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.080996 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.082213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.083074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.083509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.099448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:56.106788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2122816-22dd-486a-8484-fe8cb7617cb6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:56.109404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:56.110524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:24:56.111536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:56.114909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 429/854] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:56.123799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.151790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.152154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.152443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.152723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.155423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:56.164965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.166454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:56.185304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:56.186864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:56.187982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:56.187982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:56.187982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:56.187982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:56.187982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 426/855] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:56.192305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.235683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:56.255144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67b7bd-fce9-499a-a1ab-1e9b0ea90bb1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.291133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.291834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.292124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.292402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.293151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:56.293462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.294841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.304602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 430/856] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:56.307572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66274c27-6df5-4f29-b284-e8a4b3e7ab56 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.309674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.346839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.347593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.347906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.348174 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.348896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:56.350041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.350671 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.367863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 427/857] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:56.368740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.418633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348c6a6-5f5d-4c51-8055-ed1639cf9b82 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.419685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:56.454717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.455886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.456399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.456884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.468210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.469959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.476832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:56.478512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38722dfa-daf3-4036-b447-a63d77e006c0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:56.488184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:56.494739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:56.495615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:56.495615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:56.495615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:56.495615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:56.495615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 431/858] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:56.495615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.517700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.518715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.519171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.520157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.528263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.530179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:56.545834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:56.546921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:56.547993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:24:56.554579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 428/859] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:24:56.555588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.601110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a5fa0-335c-4ed7-86dd-14a8de897853 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.635869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.636832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.636832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.637065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.637796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:56.638105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.639728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.653487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 432/860] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:56.660678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.665470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76edb6-cf2a-450d-9eac-69135011945d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.673897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:56.701645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.702301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.702584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.702962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.703755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:56.704482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.706570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.720579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 429/861] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:56.733727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.738891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:56.781923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c643c42-4e6e-451d-be4a-9ef9929f0604 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.821846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.822820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.823169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.823532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.832708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.834592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:24:56.850261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:56.851723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:56.852782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 433/862] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:56.859341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:56.863811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c153279-52a0-4113-84f5-44abc6f297d8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.904718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:56.905840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:56.906340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:56.906834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:56.907892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=9196a10f723649839a7eca20e99d84f4) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:56.908745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:56.910507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:56.917389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 430/863] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/role_assignments?user.id=9196a10f723649839a7eca20e99d84f4&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:56.970133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d35fe-6ee2-4ad0-ae5f-41c231dfc5c3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:56.990146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:57.009015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:57.009693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.010038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.010315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.011282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:57.011598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:57.013080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:57.034123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 434/864] 10.222.0.41 () {56 vars in 1135 bytes} [Mon Jun 6 20:24:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:57.039869 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:57.074831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:57.153651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e787c3a-4672-4bad-bc88-8177748085a6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:57.191046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:57.191852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.192064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.192336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.267371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:57.269386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:57.300856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:57.378607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 431/865] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:24:57 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:24:57.383947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:57.489477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd099a-badd-45c0-a17d-925122c28c27 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:57.532637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:57.533698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.533983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.534261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.544934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:57.546939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:24:57.565137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:57.566073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:57.566956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:24:57.566956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:24:57.566956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:24:57.566956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:57.566956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 435/866] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:57.573344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:57.674260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a60d84-5034-48d3-9c87-eb799b8a9879 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:57.703546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:57.709786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:57.723885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.723885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.723885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.723885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:57.723885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:57.723885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b80a8-6195-4cb3-8875-fc38b897b632 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:57.726914 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 432/867] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:24:57.733352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:57.762834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:57.840879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f512c-6ef2-4cc6-86c1-8ce615b0c1a3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:57.845070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81697b11-e190-41f3-9a6a-ab3f27059659 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.845356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81697b11-e190-41f3-9a6a-ab3f27059659 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.845637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81697b11-e190-41f3-9a6a-ab3f27059659 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.848402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 433/868] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:24:57.852979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58272566-d775-4eb3-846c-28b52d95009b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.853264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58272566-d775-4eb3-846c-28b52d95009b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.853538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58272566-d775-4eb3-846c-28b52d95009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.875931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-58272566-d775-4eb3-846c-28b52d95009b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:57.877386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-58272566-d775-4eb3-846c-28b52d95009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:57.880811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:57.881678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:57.882030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:57.882375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:57.889561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:57.891104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:24:57.905698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:57.906633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:57.907636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 436/869] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:24:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:57.919855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:57.957775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 434/870] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:57.979976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58272566-d775-4eb3-846c-28b52d95009b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:58.025506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:58.042036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36783db9-8c3d-4130-82db-1823206278a9 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:58.084479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:58.085173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:58.085460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:58.085741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:58.086530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:58.086843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:58.088303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:58.105383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58272566-d775-4eb3-846c-28b52d95009b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:58.105877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 437/871] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:24:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:58.106938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:58.129731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:58.146366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4759a2d-63c5-4740-b5b2-e3770f9a1a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:58.147309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4759a2d-63c5-4740-b5b2-e3770f9a1a20 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:58.147779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4759a2d-63c5-4740-b5b2-e3770f9a1a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:58.148316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4759a2d-63c5-4740-b5b2-e3770f9a1a20 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:58.149191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4759a2d-63c5-4740-b5b2-e3770f9a1a20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:58.150804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4759a2d-63c5-4740-b5b2-e3770f9a1a20 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:58.211017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee04cf-700a-4a76-9a02-51db39236109 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:58.225416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 435/872] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:24:57 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:24:58.278734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:58.280388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:58.280906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:58.281402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:58.282464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=e168d14c0eac4ee2a5acdc23c39118d9) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:58.282986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:58.285098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de3eef-aaeb-4a44-9567-0c47d85a3b17 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:58.293235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 438/873] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:24:58 2022] GET /identity/v3/role_assignments?user.id=e168d14c0eac4ee2a5acdc23c39118d9&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:58.375925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a556c-2285-44c3-b7e6-c352799b8c7f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:58.376587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a556c-2285-44c3-b7e6-c352799b8c7f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:58.377072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a556c-2285-44c3-b7e6-c352799b8c7f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:58.378752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 436/874] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:58.385121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c7675-08f8-430b-a58f-dd7860ed4421 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:58.386156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c7675-08f8-430b-a58f-dd7860ed4421 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:58.386684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c7675-08f8-430b-a58f-dd7860ed4421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:58.407131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-499c7675-08f8-430b-a58f-dd7860ed4421 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:58.409244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-499c7675-08f8-430b-a58f-dd7860ed4421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:58.411964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:58.487382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 439/875] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:58 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:24:58.513242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a556c-2285-44c3-b7e6-c352799b8c7f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:58.620177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a556c-2285-44c3-b7e6-c352799b8c7f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:58.655653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6252e6-b7fc-4ca8-a25f-ca586edb7e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:58.656384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6252e6-b7fc-4ca8-a25f-ca586edb7e64 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:58.656698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6252e6-b7fc-4ca8-a25f-ca586edb7e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:58.656984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6252e6-b7fc-4ca8-a25f-ca586edb7e64 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:58.657707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6252e6-b7fc-4ca8-a25f-ca586edb7e64 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:58.659327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6252e6-b7fc-4ca8-a25f-ca586edb7e64 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:58.754667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 437/876] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:24:58 2022] POST /identity/v3/services => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:24:59.419341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52779e9-ef82-4440-94e7-a1b646ea9481 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:59.419909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52779e9-ef82-4440-94e7-a1b646ea9481 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:59.419909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52779e9-ef82-4440-94e7-a1b646ea9481 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:59.422144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 440/877] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:24:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:24:59.427619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:59.428080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:59.428254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:59.445831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:24:59.447055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:24:59.483720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:59.523065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 438/878] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:24:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:24:59.547690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52779e9-ef82-4440-94e7-a1b646ea9481 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:59.656854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52779e9-ef82-4440-94e7-a1b646ea9481 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:59.692233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:59.693035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:59.693320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:59.693595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:59.700613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:24:59.705078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:24:59.713337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:24:59.718619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:24:59.719458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 441/879] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 20:24:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:24:59.721404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:59.819973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a677ba-f51f-45e5-9c1b-495507604254 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:24:59.856003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:24:59.856664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:24:59.856943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:24:59.857281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:24:59.858177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:24:59.858525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:24:59.859951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:24:59.868409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 439/880] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:24:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:24:59.873565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:24:59.979768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8824cf-7beb-4da1-a3d9-de69ac15605b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:00.016082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:00.016878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.017160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.017433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.029306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:00.030805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:00.048683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:00.049611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:00.050499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:00.050499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:00.050499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:00.050499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:00.050499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 442/881] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:24:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:00.055515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:00.089203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:00.162977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75f0d0-632f-4a67-90c3-c083be384cd3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:00.197465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:00.198120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.198406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.198681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.199401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:00.199734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:00.201097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:00.213958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 440/882] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:00.219208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:00.278090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:00.334687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e8333-37cd-45d3-8658-87c0ba9847ec None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:00.369603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:00.370336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.370620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.370897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.379541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:00.381027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:00.396381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:00.397383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:00.398315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 443/883] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:25:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:00.405379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:00.499398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:00.511396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc275494-ca8b-48e9-98d5-9465f895bdf2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:00.546689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:00.547355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.547636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.547940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.548778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:00.549090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:00.550510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc3191-65cf-4c63-87dd-8994a4dd3512 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:00.569398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 441/884] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 20:25:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:00.576772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:00.660269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611518ac-576c-4288-8378-23f4cf400348 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.660512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611518ac-576c-4288-8378-23f4cf400348 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.660770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611518ac-576c-4288-8378-23f4cf400348 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.662555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 442/885] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:00.668686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2a643-d03d-4cbf-bd88-57ba759527ed None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.668975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2a643-d03d-4cbf-bd88-57ba759527ed None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.669256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2a643-d03d-4cbf-bd88-57ba759527ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.684626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c682c5b-c60b-476d-9345-e3ddb8cbb1e6 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:00.690557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bac2a643-d03d-4cbf-bd88-57ba759527ed None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:00.691777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bac2a643-d03d-4cbf-bd88-57ba759527ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:00.697061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:00.727275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:00.728406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.729038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.729521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.778436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 443/886] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:00.781031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6055e-4331-4ab1-a8c0-f2db617d9e98 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.781325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6055e-4331-4ab1-a8c0-f2db617d9e98 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.781606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6055e-4331-4ab1-a8c0-f2db617d9e98 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.787233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 444/887] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:00.790023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:00.790023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:00.790023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:00.811639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:00.813102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:00.818208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:00.819676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:00.884781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 445/888] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:00.885613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:00.896127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:00.929037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 444/889] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:00 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:25:00.930199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:00.943474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:00.997265 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dfe1c2-0e7d-4870-963b-d93cad558e51 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.037668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.038480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.038764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.039036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.039701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.040059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad7c7f-8208-475c-9a80-5b3e203f0429 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.041072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:01.053534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:01.054754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:25:01.055840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.059824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 446/890] 10.222.0.41 () {56 vars in 1132 bytes} [Mon Jun 6 20:25:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:01.061217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.079833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.080639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.080929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.081207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.081854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.083224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:01.094999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:01.101067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:01.102194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 445/891] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:01.103183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.130333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.160797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eba55c-06a4-4083-afa0-4d8cb2d4c554 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.175034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.194510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.195233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.195515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.195822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.199410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d8f359-f8e4-4a55-b9a3-e61d74ba58e2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.206087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.207524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ec3b3cd7-be2f-4b88-8c72-19184b837480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:01.224115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:01.225016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:01.225915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:01.225915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:01.225915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:01.225915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.225915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 447/892] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:01.226841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9501-4e4c-410a-becd-95f2d83316be None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.227123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9501-4e4c-410a-becd-95f2d83316be None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.227403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118c9501-4e4c-410a-becd-95f2d83316be None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.228836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 448/893] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:01.229384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9501-4e4c-410a-becd-95f2d83316be None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.241750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.242410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.242695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.242969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.243708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.244006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.245382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.250457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 446/894] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.251214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.331710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118c9501-4e4c-410a-becd-95f2d83316be None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.348629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf724f-92ee-49f7-b84b-65d18addab5a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.353540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.365172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.365823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.372004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.372004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.372004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.372004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.372004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244b9439-fc78-4a5e-a8ce-a230ab6cc550 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.373839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 449/895] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.375084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.375366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.375643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.382885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.383550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.383868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.384150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.384873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.385183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.386542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.392908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:01.393614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.394083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:01.398568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 447/896] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.399340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.460794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 450/897] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:01.461712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.499329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72404ac4-7301-454a-a097-6d2389198544 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.533392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.534054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.534344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.534625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.535421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.535777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.537152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.542200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 448/898] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.543074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.566139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3842fe-3082-49d9-a4b5-0bb633d55643 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.582544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.600527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.601188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.601448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.601871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.601871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.602409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.602718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.604103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.609274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 451/899] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.610037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.645314 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f64c1d-9bef-48f1-98f7-7cde5ce43926 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.684409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.685223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.685514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.685839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.693630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.695106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:01.709887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:01.710909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 449/900] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:01.712260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.742648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb92cc-84e2-47de-8074-862c7d46bf70 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.781481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.782289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.782572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.782848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.783811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.785214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:01.798255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:01.799308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:01.800521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 452/901] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:01.801617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.835901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253add7-8252-4fe4-a42c-387651def32c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.850836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.860398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:01.871075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.871761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.872050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.872329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.873110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] RBAC: Adding query filter params (interface=public, service_id=14287aa661a64c9e83796307c8294e99, region_id=RegionOne) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.873464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.874821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.880264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 450/902] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/endpoints?service_id=14287aa661a64c9e83796307c8294e99&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.881065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.912541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8ae8c-ae7e-4a35-994d-ce3b508945bb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:01.945479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:01.946135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:01.946419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:01.946694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:01.947472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:01.947937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:01.949325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:01.967783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 453/903] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:01.967783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:01.992070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43921a-c1f1-4591-9986-73667019672a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:02.028799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:02.029524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:02.030303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:02.030586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:02.031278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:02.031596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:02.032980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:02.037944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 451/904] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:02.038720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:02.081807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18908ac-9868-4d24-9325-1135a7615c07 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:02.084934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:02.096242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:02.118348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:02.119559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:02.120203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:02.120830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:02.121689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:02.123343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:02.139704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 454/905] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:25:01 2022] GET /identity/v3/services => generated 1268 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:25:02.139704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:02.148443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbcae6-7317-4da2-9f43-7440f9507428 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:02.186123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:02.186833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:02.187114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:02.187484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:02.188538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] RBAC: Adding query filter params (role.id=8d9ef90e4b434f55ba2f803850f15085, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=9196a10f723649839a7eca20e99d84f4) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:02.188848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:02.190369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:02.197546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 452/906] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:02 2022] GET /identity/v3/role_assignments?user.id=9196a10f723649839a7eca20e99d84f4&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=8d9ef90e4b434f55ba2f803850f15085 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:02.271538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b7be34-7911-4887-b6cd-7bce175c0799 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:02.313062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:02.313750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:02.314038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:02.314318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:02.315058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:02.315369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:02.316932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:02.323101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 455/907] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:25:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:02.329909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:02.444218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65757a4-c8b0-4cb3-be6a-2f9fa21ca309 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:02.458851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:02.478844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:02.479554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:02.479813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:02.480090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:02.480867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] RBAC: Adding query filter params (interface=public, service_id=26304f1896ec4f63a7002c6d4525d0b9, region_id=RegionOne) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:02.481173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:02.482556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44b641-6557-4913-a68e-93990cb11ee1 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:02.488038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 453/908] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:25:02 2022] GET /identity/v3/endpoints?service_id=26304f1896ec4f63a7002c6d4525d0b9&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:02.493100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:02.527402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:02.599653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd13e0c-39d4-4ddf-89a4-60ee9fe8680a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:02.636245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261aa77a-f416-4d2c-904c-96cd0695a559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:02.636906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261aa77a-f416-4d2c-904c-96cd0695a559 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:02.637194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261aa77a-f416-4d2c-904c-96cd0695a559 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:02.637474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261aa77a-f416-4d2c-904c-96cd0695a559 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:02.638154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261aa77a-f416-4d2c-904c-96cd0695a559 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:02.639558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261aa77a-f416-4d2c-904c-96cd0695a559 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:02.645873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 456/909] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:25:02 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:25:04.144551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aee72e-cc88-4c30-a894-c71d11dcdd12 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.145102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aee72e-cc88-4c30-a894-c71d11dcdd12 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.145102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aee72e-cc88-4c30-a894-c71d11dcdd12 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.146738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 454/910] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:04.152071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1513c8-6ccd-4b44-9a06-14b39b0651a5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.152474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1513c8-6ccd-4b44-9a06-14b39b0651a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.152658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1513c8-6ccd-4b44-9a06-14b39b0651a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.169940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0d1513c8-6ccd-4b44-9a06-14b39b0651a5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:04.171711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0d1513c8-6ccd-4b44-9a06-14b39b0651a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:04.246155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 457/911] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:04.269032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aee72e-cc88-4c30-a894-c71d11dcdd12 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:04.384108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aee72e-cc88-4c30-a894-c71d11dcdd12 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:04.418644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ba819-2c6b-4e0b-ab68-95d3f47ae0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:04.419630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba819-2c6b-4e0b-ab68-95d3f47ae0b6 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.420683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba819-2c6b-4e0b-ab68-95d3f47ae0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.421164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ba819-2c6b-4e0b-ab68-95d3f47ae0b6 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.422117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ba819-2c6b-4e0b-ab68-95d3f47ae0b6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:04.423706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ba819-2c6b-4e0b-ab68-95d3f47ae0b6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:04.494080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 455/912] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:25:04 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:04.661415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b52cbd-050d-4f05-afc2-bcb69397c93b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.661814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b52cbd-050d-4f05-afc2-bcb69397c93b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.662014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b52cbd-050d-4f05-afc2-bcb69397c93b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.663713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 458/913] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:04.669493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.669785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.670007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.686776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfd326-bba1-4abf-a160-68ef78f0dcde None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.687125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfd326-bba1-4abf-a160-68ef78f0dcde None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.687335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfd326-bba1-4abf-a160-68ef78f0dcde None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.688189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:04.689420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:04.689811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 459/914] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:04.695147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.695562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.695983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.714190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:04.715518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:04.726184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:04.739252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:04.783532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 456/915] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:04.805655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 460/916] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:04.811914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:04.832077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:04.946121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba33a00-84a4-4e42-9b30-e2a22e301639 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:04.950935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59798ce-083b-46cf-bacc-94bae64b402b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:04.983138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:04.984447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.984962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.985441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:04.986290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:04.987911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:04.995066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:04.995892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:04.996174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:04.996595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:05.002347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:05.003272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 06 20:25:05.004171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:05.007069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 461/917] 10.222.0.41 () {56 vars in 1132 bytes} [Mon Jun 6 20:25:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:05.012893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.013832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:05.022180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:05.023150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:05.024035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:05.028206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 457/918] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 20:25:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:05.029559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.045922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.057660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.139644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca04af-00c4-4d14-9089-453446476146 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.151742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113de0d-c8ae-4a42-8f01-95c46e465331 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.177530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.178254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.178543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.178824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.179571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:05.179922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.181428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae29dd-dd7d-42ab-8405-c5c5fa678e0e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.194957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 458/919] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:05.196583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbfbaf-5942-4346-bf9b-653621219ff6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.197011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbfbaf-5942-4346-bf9b-653621219ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.197219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edbfbaf-5942-4346-bf9b-653621219ff6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.198599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 459/920] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:05.199171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbfbaf-5942-4346-bf9b-653621219ff6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.201251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.201909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.202190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.202469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.203161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:05.203680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.205889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee647b-a78a-4ba9-b002-dce7a8632225 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.211485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 462/921] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:05.215883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.215883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.215883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.234718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:05.235969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:05.302168 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.315730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 463/922] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:05.315730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.326562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edbfbaf-5942-4346-bf9b-653621219ff6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.335949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.364937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.366084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.366384 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.366651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.377439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.378903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:05.396900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:05.397844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:05.398809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:05.398809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:05.398809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:05.398809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:05.398809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 460/923] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:05.399694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.428180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900202e6-784c-4080-9fa0-df4da8df14b8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.462149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.462737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.463018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.463291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.464192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:05.464497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.466331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.476173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 464/924] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:05.476173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.506570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684819e-9173-4b88-9238-ae35a63158f8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.544308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.545723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.546012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.546290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.546931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.548681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.557543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:05.560151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:05.561166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:05.561988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 461/925] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:05.567049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.570138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.581827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2725b1ca-58fa-4307-aeba-f8b4cc6cddd4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.620956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.621767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.622049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.622411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.623158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:05.623491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.624882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.642136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 465/926] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:05.642136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.676795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3988e46-4bba-4376-b848-077d08ac11b6 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.713435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.714019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.714302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.714580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.715316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.716720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.744528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f575f6-4a47-4d7f-8225-161f945f7922 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.777674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.778382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.778669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.778941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.780310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:05.780618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.781986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.786814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 466/927] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:05.787571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.792034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.811379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 462/928] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:25:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:05.812408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:05.895891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:05.908734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e340943-fa64-4b4b-bde4-ca2f79aba93e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.955567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:05.956963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:05.957550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:05.957731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787b6d1-a482-435c-84ce-b1064cf05c01 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:05.958226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:05.968427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:05.970306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:05.989253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:05.990251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:25:05.991186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:05.998938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 467/929] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:06.000583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.002989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.004036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.004326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.004609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.006340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:06.006654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.008063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.014876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 463/930] 10.222.0.41 () {56 vars in 1114 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:06.020247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.075964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:06.137160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245bfc1-aa9e-4d73-8b44-def56b660a4b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:06.147366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1823d79-bd76-4ac0-81b8-ce70b8919452 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:06.165496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:06.182910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.183584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.184041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.184574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.185338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.185461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.186820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.187283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.187775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.187879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.188886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:06.189376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.191023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.212276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 468/931] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 20:25:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:06.217791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.286598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 464/932] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:25:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:06.342387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7388e04-9b5f-49ac-9756-bee5a703e70b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:06.364691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:06.395327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.396617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.397137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.397640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.411218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.413154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:06.433444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:06.434412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:06.435383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:06.435383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:06.435383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:06.435383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:06.438494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 469/933] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:06.444012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.556470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f81979-0313-4da6-a7c3-650b371dc65b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:06.591452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.592194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.592521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.592806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.593532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:06.593841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.595216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.608414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 465/934] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:06.613484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.717519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a20004-9aa2-4ad6-b06a-1bd88db87842 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:06.759025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.759846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.760131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.760406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.767541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.769019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-46141909-1169-4f99-ab3b-676e7e2b5fa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:06.783602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:06.784521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:06.786353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 470/935] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:06.790990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.826912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:06.833152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135b86e-0fa8-49b3-8b5e-8032697ce29a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.833493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135b86e-0fa8-49b3-8b5e-8032697ce29a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.833785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135b86e-0fa8-49b3-8b5e-8032697ce29a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.835111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 471/936] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:06.842495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.842784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.843057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.864404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:06.865918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:06.884459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:06.907152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0831f5e-4e9b-4a81-98b6-3c522a51b380 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:06.942611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:06.943413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:06.943761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:06.944061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:06.944838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:06.945150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:06.946030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 472/937] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:06.946582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:06.958833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 466/938] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:25:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:06.964559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:06.969634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:07.066570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:07.070409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d660b1b-5194-43a2-b9f7-5f4f28a10c48 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:07.074063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8da812-45cd-4973-b10a-0d7fbf7ed800 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:07.106698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:07.108101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:07.108389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:07.108665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/9196a10f723649839a7eca20e99d84f4/roles/8d9ef90e4b434f55ba2f803850f15085` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:07.111188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=9196a10f723649839a7eca20e99d84f4, role_id=8d9ef90e4b434f55ba2f803850f15085)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:07.112491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:07.112803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:07.113439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:07.113798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:07.114150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:07.115155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:07.117357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:07.130188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:07.131536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:07.132434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 467/939] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:07.139701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:07.189151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 473/940] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:25:06 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/9196a10f723649839a7eca20e99d84f4/roles/8d9ef90e4b434f55ba2f803850f15085 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:25:07.254990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f708e52-a230-4ef1-aeda-4492a80c8333 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:07.295627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:07.296343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:07.296654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:07.296905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:07.297762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:07.298078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:07.299516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:07.305615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 468/941] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:07.311357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:07.332242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:07.420645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd014e41-3f51-46c9-87c4-8d836283e3d7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:07.456568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:07.457232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:07.457515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:07.457791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:07.458493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:07.458800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:07.460199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:07.465592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 474/942] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:07.472003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:07.497364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:07.577008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572cb87-adbd-4bae-bc18-1be1db2db79e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:07.613488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:07.614535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:07.615025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:07.615500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:07.616525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] RBAC: Adding query filter params (interface=public, service_id=83959f727cfc4b01a9db92522a142585, region_id=RegionOne) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:07.617057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:07.618653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919c64c3-380b-45a6-81c9-42e58c47b542 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:07.625296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 469/943] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:25:07 2022] GET /identity/v3/endpoints?service_id=83959f727cfc4b01a9db92522a142585&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:07.633518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:07.711933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:07.733689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd42acb-e302-4697-a2d9-5b5d19134787 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:07.769521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58cd49f-5f7c-4157-ae25-fc061f3fa020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:07.770194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cd49f-5f7c-4157-ae25-fc061f3fa020 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:07.770477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cd49f-5f7c-4157-ae25-fc061f3fa020 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:07.770753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58cd49f-5f7c-4157-ae25-fc061f3fa020 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:07.771436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cd49f-5f7c-4157-ae25-fc061f3fa020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:07.772834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58cd49f-5f7c-4157-ae25-fc061f3fa020 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:07.779274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 475/944] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:25:07 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:25:08.264027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118944e9-e7b7-481d-b73a-92bb234b12c5 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.265002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118944e9-e7b7-481d-b73a-92bb234b12c5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.265490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118944e9-e7b7-481d-b73a-92bb234b12c5 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.267396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 470/945] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:08.273333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9718-6749-4438-9443-190f3057665c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.273882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9718-6749-4438-9443-190f3057665c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.274383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae9718-6749-4438-9443-190f3057665c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.292152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8bae9718-6749-4438-9443-190f3057665c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:08.296921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8bae9718-6749-4438-9443-190f3057665c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:08.370766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 476/946] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:08.393128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118944e9-e7b7-481d-b73a-92bb234b12c5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:08.498487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:08.508208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118944e9-e7b7-481d-b73a-92bb234b12c5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:08.543259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:08.544092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.544376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.544646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.545939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:08.547316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:08.559447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:08.560447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:08.561311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:08.564748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 471/947] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:08.568799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae9718-6749-4438-9443-190f3057665c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:08.640267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:08.675056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae9718-6749-4438-9443-190f3057665c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:08.710527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:08.711276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.711635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.712009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.712788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:08.713170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:08.714603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd983e27-28ec-456f-a74f-769760f6d2a3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:08.720364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 477/948] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:08.725782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:08.808054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49515781-014d-4083-acad-35af35fd8e35 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.808363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49515781-014d-4083-acad-35af35fd8e35 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.808551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49515781-014d-4083-acad-35af35fd8e35 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.809900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 478/949] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:08.817748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.818044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.818328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.828755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eee282-1756-499d-871a-99a62176fdf2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:08.840577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:08.842499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:08.869021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:08.869883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:08.870279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:08.870622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:08.871395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:08.871912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:08.873385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:08.887294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 472/950] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:08.929651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 479/951] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:08.955413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:08.975415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:09.072176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed86901-bd4b-4343-bbc8-916eca59a111 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:09.108505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:09.109502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.110050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.110540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.111421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:09.113060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:09.114647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING keystone.server.flask.common [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 06 20:25:09.161294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.sql.core [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 20:25:09.161294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 20:25:09.161294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [parameters: {'id': '60fe302122a246c1a653dfbb55bfec21', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 20:25:09.161294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80392) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60fe302122a246c1a653dfbb55bfec21.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60fe302122a246c1a653dfbb55bfec21. Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application conn.query(q) Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 20:25:09.179153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result.read() Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 06 20:25:09.180238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application next(self.gen) Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application yield resource Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application next(self.gen) Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise self.value Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application session.commit() Jun 06 20:25:09.181182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.session.flush() Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application compat.raise_( Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 20:25:09.182062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 20:25:09.182957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application conn.query(q) Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result.read() Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 20:25:09.183805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [parameters: {'id': '60fe302122a246c1a653dfbb55bfec21', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:09.185092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:09.186552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 60fe302122a246c1a653dfbb55bfec21. Jun 06 20:25:09.187517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.190009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 473/952] 10.222.0.41 () {60 vars in 1140 bytes} [Mon Jun 6 20:25:08 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:09.196174 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:09.268119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:09.303458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e74eab-6743-4d69-ab91-706394cd34a8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:09.345930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:09.346645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.346928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.347258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.354108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:09.355701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-26dc0db5-fb81-4350-93b1-ed118581b40c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:09.370257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:09.371858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:09.373604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 480/953] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:09.380269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:09.457023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:09.489123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7aa6c8-3c74-4658-bcd7-b9550a4a37cd None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:09.499509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e9bba-90ff-4b44-8345-5ac4202824f0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.500427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e9bba-90ff-4b44-8345-5ac4202824f0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.500921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e9bba-90ff-4b44-8345-5ac4202824f0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.503488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 481/954] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:09.508644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.509126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.509576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.527646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:09.528340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.528628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.528912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.530231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:09.530542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:09.533757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:09.534469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:09.536186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:09.558813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 474/955] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:09 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:09.620128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 482/956] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:09.643881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:09.757988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4865a69-f54e-469b-b92e-3d63e2671f8d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:09.789384 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:09.796670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:09.797504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.797785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.798058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.810169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:09.818949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:09.820031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:09.820911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 475/957] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 20:25:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:09.827006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:09.892365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:09.937322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e1054-1ff2-45ff-9a7f-2743ba344267 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:09.973415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:09.974081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:09.974370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:09.974650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:09.975393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:09.975726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:09.977096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20526a7-e9b1-4de9-833e-626c0387f070 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:09.985648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 483/958] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:25:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:09.990767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.096124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ba9fc-af6c-4e0a-b60e-f25d58489e86 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.132143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.133291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.133765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.134224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.146146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.147988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:10.166249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:10.167182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:10.168239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:10.168239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:10.168239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:10.168239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:10.170783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 476/959] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:10.175692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d0323-7170-4f46-8048-8c2017d55764 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.175999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d0323-7170-4f46-8048-8c2017d55764 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.176211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.176316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d0323-7170-4f46-8048-8c2017d55764 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.177883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 484/960] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:10.183044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.183342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.183619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.201497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:10.202695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:10.224502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:10.299688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0019bd-63dc-41cc-8911-2f7fecf73bf4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.299892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 485/961] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:10.324088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.336041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.336987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.337427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.337856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.338918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:10.339378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.340944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.354942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 477/962] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:10.360656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.383686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:10.442595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c2819-8571-4d5b-99f0-22e901327c37 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.477181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a5dbd-4d08-4c7b-bcef-da137113a5f3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.479599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:10.479750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.481680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.481680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.482022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.483313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.484822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:10.497333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:10.498469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:10.499354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:10.503884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 486/963] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:10.504918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.520975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.522185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.522711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.523216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.532157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.534397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:10.551631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:10.552797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 06 20:25:10.553745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:10.556244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 478/964] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:10.561446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.628959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf31615-d312-456d-89ce-47be58bc5c9b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.646040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:10.670449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b773f1-ed00-4a9c-98f2-25574be1e287 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.673642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.674525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.674983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.675433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.676331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:10.676837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.678782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.685192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 487/965] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:10.691857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.719504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.720538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.721037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.721523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.723068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:10.723867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.726235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.755604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 479/966] 10.222.0.41 () {56 vars in 1149 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:10.757103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.787062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:10.809915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594aec07-9e68-4298-a572-0d3cf45fe9fa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.848635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.849755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.849973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.850355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.851195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:10.851630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:10.853160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:10.864895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 488/967] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:10.865998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:10.895140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd043fd-36c8-475f-8ff5-524f62776fb6 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:10.917853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:10.946793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78688e99-2e68-49f6-9467-beec1ff8bc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:10.948557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78688e99-2e68-49f6-9467-beec1ff8bc09 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:10.948849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78688e99-2e68-49f6-9467-beec1ff8bc09 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:10.949140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78688e99-2e68-49f6-9467-beec1ff8bc09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:10.984205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfa28d-c4ee-406e-8742-4cc046eec0da None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:11.027166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:11.028577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.030063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.030833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.032074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.034674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.042690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78688e99-2e68-49f6-9467-beec1ff8bc09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.044108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78688e99-2e68-49f6-9467-beec1ff8bc09 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.065782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:11.116973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 489/968] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:25:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:11.160882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 480/969] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:10 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 20:25:11.166292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:11.288045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:11.295563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093a46c-a060-4e6c-8fb5-46d0982c13f7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:11.297433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f402419-03e8-4967-9b4f-a6db342031a3 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.297724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f402419-03e8-4967-9b4f-a6db342031a3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.298008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f402419-03e8-4967-9b4f-a6db342031a3 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.299579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 481/970] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:11.305712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.306330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.306839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.328702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:11.330771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:11.342023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:11.342947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.343254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.343691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.355795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.357252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:11.377752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:11.384967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:11.386063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:11.386063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:11.386063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:11.386063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:11.386063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 490/971] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:11.387330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:11.431897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 482/972] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:11.458160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:11.504430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d82811-ee91-494c-8d3e-4f49a45c7adf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:11.516470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:11.557722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:11.558699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.559183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.559638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.561668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:11.562162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.564249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.578482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e5c8f-8acd-4a0e-93a2-90c286cbef06 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:11.584253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 491/973] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:11.585319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:11.590027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:11.626087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:11.627008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.627481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.627924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.628792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.630310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.703866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25bc57-db66-4c22-91a0-cbe4cd9899a0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:11.707021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 483/974] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:25:11 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:11.752499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:11.753622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.754105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.754526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.763015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.764873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:11.781472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:11.782648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:11.784043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:11.786281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 492/975] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:11.792368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:11.905997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc945de-5ccd-4213-81a9-0a74065e04cd None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:11.917747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:11.943769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:11.944678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:11.945143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:11.945578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:11.946613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:11.947077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:11.948716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ef8575-83e7-490c-bdab-484b5bca23be None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:11.963149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 484/976] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:25:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:11.970134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:12.013969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994dd5a-292c-4617-b519-b276d42dc2ca None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:12.014288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994dd5a-292c-4617-b519-b276d42dc2ca None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:12.014529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994dd5a-292c-4617-b519-b276d42dc2ca None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:12.016219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 485/977] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:12.021299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:12.021583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:12.021861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:12.039335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:12.040558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:12.041778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:12.086864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc843f8a-316b-42bd-aa5e-87d947b40519 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:12.115982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 486/978] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:12.129782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:12.130780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:12.131260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:12.131800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:12.132903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] RBAC: Adding query filter params (role.id=8d9ef90e4b434f55ba2f803850f15085, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=9196a10f723649839a7eca20e99d84f4) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:12.133425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:12.135104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:12.141129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:12.142246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 493/979] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:11 2022] GET /identity/v3/role_assignments?user.id=9196a10f723649839a7eca20e99d84f4&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=8d9ef90e4b434f55ba2f803850f15085 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:12.258400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:12.261875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5353255-89e4-42a7-b3aa-b3c688d22923 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:12.300764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:12.301455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:12.301741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:12.302265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:12.302824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:12.304729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:12.343183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.sql.core [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 20:25:12.343183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 20:25:12.343183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [parameters: {'user_id': 'c40aa8e9f71e4815aeba3c50beff6c2a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 20:25:12.343183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80392) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application conn.query(q) Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result.read() Jun 06 20:25:12.353006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 20:25:12.354263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application next(self.gen) Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application yield resource Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application next(self.gen) Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.session.rollback() Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.force_reraise() Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise self.value Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application session.commit() Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 06 20:25:12.355285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._prepare_impl() Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.session.flush() Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._flush(objects) Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application compat.raise_( Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application flush_context.execute() Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rec.execute(self) Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 06 20:25:12.357875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 06 20:25:12.361470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result = self._query(query) Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application conn.query(q) Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application result.read() Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 06 20:25:12.362319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c40aa8e9f71e4815aeba3c50beff6c2a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:12.363139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 06 20:25:12.364003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:12.365043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 487/980] 10.222.0.41 () {60 vars in 1131 bytes} [Mon Jun 6 20:25:12 2022] POST /identity/v3/users => generated 160 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:12.365429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:12.472443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1090bb6-10f6-4e4d-8261-db9aa648fdcf None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:12.496728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:12.508479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:12.510583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:12.510583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:12.510583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:12.517842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:12.519404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-8581ab2e-17bd-4c0e-bd8c-87a6d2eeaa1f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:12.534798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:12.535955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 06 20:25:12.537112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:12.539813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 494/981] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:12.546633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:12.651956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517386-f863-4b31-ae70-51fe49d9b255 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:12.687486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:12.688403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:12.688840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:12.689267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:12.690235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:12.690692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:12.692493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b01f2c-0d69-4a01-884a-7ea44a25e357 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:12.730662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 488/982] 10.222.0.41 () {56 vars in 1101 bytes} [Mon Jun 6 20:25:12 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:13.531452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4e05-e8a1-47ba-8494-0ff1851b6f3b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:13.532015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4e05-e8a1-47ba-8494-0ff1851b6f3b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:13.532015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff4e05-e8a1-47ba-8494-0ff1851b6f3b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:13.533734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 495/983] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:13.539267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:13.539559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:13.539871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:13.557163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:13.558681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:13.574250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:13.627198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 489/984] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:13.650145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4e05-e8a1-47ba-8494-0ff1851b6f3b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:13.754348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff4e05-e8a1-47ba-8494-0ff1851b6f3b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:13.787966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:13.788686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:13.788968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:13.789243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:13.789886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:13.791249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:13.803204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:13.804426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:13.805279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:13.807785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 496/985] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:13.811351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:13.912353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42245f83-5cf6-4fe3-9219-c1fa760a3e43 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:13.946876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:13.947701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:13.947998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:13.948276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:13.948974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:13.949281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:13.950654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f086a-e6af-4cc1-b050-965a87e39a8e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:13.959979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 490/986] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:13.961532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.012410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.031566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e682-d06a-4374-aeff-402416fd7d8d None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.031911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e682-d06a-4374-aeff-402416fd7d8d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.032151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e682-d06a-4374-aeff-402416fd7d8d None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.033800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 491/987] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:14.039238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.039464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.039800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.058059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:14.059268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:14.068048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612abe9b-5dcc-4aef-869b-7da549bfb5cd None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.069770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.104126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.104791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.105442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.105728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.106750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:14.107066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.109212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.115189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 497/988] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:14.120873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.137060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 492/989] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:14.159673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.240755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fb027-1b36-4c5b-afe9-d394aa2273e1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.253564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.273280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4676d4-c63d-4243-ae5b-92a10017d738 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.273663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.274320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.274605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.274882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.275732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] RBAC: Adding query filter params (interface=admin, service_id=83959f727cfc4b01a9db92522a142585, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:14.276070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.277448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.285768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 498/990] 10.222.0.41 () {56 vars in 1249 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/endpoints?service_id=83959f727cfc4b01a9db92522a142585&interface=admin®ion_id=RegionOne => generated 193 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:14.293857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.314906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.315722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.316015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.316286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.316935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.318309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:14.330045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:14.331044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:14.332152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 493/991] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:14.337544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.349030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.405655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e969b9-a981-4857-aefe-0844fd240582 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.441566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.442227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.442509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.442697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc6c69-d3b2-451e-9e9e-241c409f384b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.442813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.443445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.445803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.453475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 499/992] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:25:14.481488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.482177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.483037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.483316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.484090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] RBAC: Adding query filter params (name=compute) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:14.484403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.485825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21181c75-c576-469f-9955-6f6676bfadec None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.491734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 494/993] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:14.497400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.520513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6532c-152e-448b-a19f-79b678e63349 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.520842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6532c-152e-448b-a19f-79b678e63349 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.521084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6532c-152e-448b-a19f-79b678e63349 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.523453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 495/994] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:14.530055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.530363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.530991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.549238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.549906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:14.551091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:14.616357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7155c0-5ef3-40ce-8b05-6f605ac83999 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.627333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 496/995] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:14.650910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.653321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.653544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.654770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.655055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.655330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.656075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] RBAC: Adding query filter params (type=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:14.656380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.658448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.663085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 500/996] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:14.669184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.759228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b00a839-bf27-4a37-8790-2d62fe45fe2e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.772344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0a206-737b-497c-9d5e-dc00fa313dff None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.774609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.797160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.798222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.798655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.799209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.800088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.801635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.806849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.807500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.807817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.808095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.808880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] RBAC: Adding query filter params (interface=public, service_id=dc9651330c5f4dc5acf28928bde96a7d, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:14.809192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.810578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:14.814529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:14.815511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:14.816385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 497/997] 10.222.0.41 () {56 vars in 1105 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:14.820282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 501/998] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/endpoints?service_id=dc9651330c5f4dc5acf28928bde96a7d&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:14.821022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.823157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:14.890022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:14.927222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2417b6c-a206-4064-a6af-0ac40b90e7c7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.929297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e72f742-add4-4c66-a66b-b8e785329ebc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:14.966225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.967441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.967757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.968037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.968601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:14.968745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.969257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:14.969541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:14.969819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:14.970083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.970523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] RBAC: Adding query filter params (name=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:14.970833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:14.972232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760c51f-e9de-4589-be9f-fc512b848653 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:14.977029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 498/999] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:25:14.982246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 502/1000] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:14.983049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:15.094862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31943d5-de14-4136-a03a-380a5ddd63b3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:15.125079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:15.131526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:15.132303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:15.140117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:15.140117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:15.140117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] RBAC: Adding query filter params (type=image) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:15.140117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:15.140117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85410fc3-e283-42b0-875a-53635905b3dd None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:15.147955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 499/1001] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:16.724540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1b9dc-9cf1-4790-a258-35714c22b1f0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:16.725045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1b9dc-9cf1-4790-a258-35714c22b1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:16.725124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae1b9dc-9cf1-4790-a258-35714c22b1f0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:16.726491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 503/1002] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:16.731749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:16.732191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:16.732360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:16.749517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:16.750716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:16.819698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 500/1003] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:16.842853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1b9dc-9cf1-4790-a258-35714c22b1f0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:16.876272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:16.953295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae1b9dc-9cf1-4790-a258-35714c22b1f0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:16.986968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:16.987709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:16.987993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:16.988264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:16.988907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:16.990281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:17.002740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:17.003757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:17.009975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 504/1004] 10.222.0.41 () {56 vars in 1126 bytes} [Mon Jun 6 20:25:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:17.010782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.112118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f343db7-f07d-4aaf-bd46-c7e19e9f324d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.136586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:17.153055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.153759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.154102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.154416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.155110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:17.155416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.156821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb543f-085a-4511-9440-0ca16db30d39 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.166967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 501/1005] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:17.168479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.234475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37c40-ae00-44ae-ad26-79ee128e28b5 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.234765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37c40-ae00-44ae-ad26-79ee128e28b5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.235047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a37c40-ae00-44ae-ad26-79ee128e28b5 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.236672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 502/1006] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:17.241467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137e202-beac-42f1-880c-562b9643735f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.241706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137e202-beac-42f1-880c-562b9643735f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.241989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6137e202-beac-42f1-880c-562b9643735f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.259067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6137e202-beac-42f1-880c-562b9643735f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:17.260280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6137e202-beac-42f1-880c-562b9643735f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:17.274309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:17.280728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ea6ad-a1ee-4460-b520-5f3147c365ca None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.315802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.316463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.316754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.317028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.317730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:17.318035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.319474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.324501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 505/1007] 10.222.0.41 () {56 vars in 1122 bytes} [Mon Jun 6 20:25:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:17.329928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.337284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 503/1008] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:17.358783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e4ab8-ef4f-477f-8f6e-249e9b6a6157 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.359068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e4ab8-ef4f-477f-8f6e-249e9b6a6157 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.359709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e4ab8-ef4f-477f-8f6e-249e9b6a6157 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.360909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 504/1009] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:17.361418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e4ab8-ef4f-477f-8f6e-249e9b6a6157 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.431989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea71fcd-fcb4-4508-b124-a27eaf77f4d2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.461360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e4ab8-ef4f-477f-8f6e-249e9b6a6157 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.464093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.464736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.465014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.465284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.466012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.467910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.478676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:17.495358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.496151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.496411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.500407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.500407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.500407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b4412793-21e8-4c11-a5f3-6f2a57ec8693 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:17.510107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:17.511256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:17.512156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 505/1010] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:17.512930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.512997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.513273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.521472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 506/1011] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:25:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:17.522274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.537811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:17.539107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:17.547964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:17.639928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d58c57-c19f-4708-ac27-c657b1b247e8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.640227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 506/1012] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:17.663474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.686194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.686867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.687156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.687423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.688167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:17.688471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.689864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.695556 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 507/1013] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:17.701789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.780325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349a0aa-56d3-4f3e-8823-e8f8f38334d5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.790318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:17.791071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:17.809080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39e159-4ff6-48bb-873a-a45902e003c1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:17.815839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a05247-a2e9-4a60-a534-e0fa4538cbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.816907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a05247-a2e9-4a60-a534-e0fa4538cbe4 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.817183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a05247-a2e9-4a60-a534-e0fa4538cbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.817457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a05247-a2e9-4a60-a534-e0fa4538cbe4 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.818546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a05247-a2e9-4a60-a534-e0fa4538cbe4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.819923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a05247-a2e9-4a60-a534-e0fa4538cbe4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.843903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:17.844739 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:17.845110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:17.845445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:17.846151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:17.846458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:17.847837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:17.852736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 508/1014] 10.222.0.41 () {56 vars in 1112 bytes} [Mon Jun 6 20:25:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:17.859075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:17.921472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 507/1015] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:25:17 2022] POST /identity/v3/services => generated 242 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:17.965698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d3027-e7f0-4f4c-8a5b-615bf84b6120 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:18.009935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40fe5-2d3b-413d-b86b-bf59602a9b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:18.010741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40fe5-2d3b-413d-b86b-bf59602a9b39 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:18.011123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40fe5-2d3b-413d-b86b-bf59602a9b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:18.011496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40fe5-2d3b-413d-b86b-bf59602a9b39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:18.012373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe40fe5-2d3b-413d-b86b-bf59602a9b39 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:18.013898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe40fe5-2d3b-413d-b86b-bf59602a9b39 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:18.022696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:18.116157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 509/1016] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:25:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:19.802462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f02949-5307-455c-b7db-5ccb65cbecc0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:19.803015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f02949-5307-455c-b7db-5ccb65cbecc0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:19.803015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f02949-5307-455c-b7db-5ccb65cbecc0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:19.804465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 508/1017] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:19.809669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:19.810052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:19.810270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:19.827133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:19.828344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:19.907190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 510/1018] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:19.930791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f02949-5307-455c-b7db-5ccb65cbecc0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.031888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f02949-5307-455c-b7db-5ccb65cbecc0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.065743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1fa3e-2165-400a-a47f-eb4b1e6490ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.066464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1fa3e-2165-400a-a47f-eb4b1e6490ec None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.066754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1fa3e-2165-400a-a47f-eb4b1e6490ec None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.067031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1fa3e-2165-400a-a47f-eb4b1e6490ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.068236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1fa3e-2165-400a-a47f-eb4b1e6490ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.069665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1fa3e-2165-400a-a47f-eb4b1e6490ec None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:20.071442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 509/1019] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:20.077505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.180825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdb594-377c-4785-9a31-8f54484561f5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.203900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:20.215718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.217139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.229108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.229108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.229108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.229871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505863a7-0fa0-4cfe-a576-8c76ba0e1661 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.229871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505863a7-0fa0-4cfe-a576-8c76ba0e1661 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.230026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:20.230099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505863a7-0fa0-4cfe-a576-8c76ba0e1661 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.230099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 510/1020] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:20.230099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4456-0933-4bdc-87b4-eaadecb54890 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.230099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4456-0933-4bdc-87b4-eaadecb54890 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.230099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaf4456-0933-4bdc-87b4-eaadecb54890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.244187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6aaf4456-0933-4bdc-87b4-eaadecb54890 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:20.245398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6aaf4456-0933-4bdc-87b4-eaadecb54890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:20.274512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:20.311706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 511/1021] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:25:20 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:20.316954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 511/1022] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:20.341314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.453084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94780205-1622-4cf2-8bc5-ffb732a98f88 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.489127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.489942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.490225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.490501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.491233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.492582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:20.499831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2089c48-5f5b-47b7-9467-e1e6903afd07 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.504025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2089c48-5f5b-47b7-9467-e1e6903afd07 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.504025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2089c48-5f5b-47b7-9467-e1e6903afd07 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.504025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 512/1023] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:20.506827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.507114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:20.507222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:20.508188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:20.509043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:20.509804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.509876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 512/1024] 10.222.0.41 () {56 vars in 1105 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:20.515542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.528186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:20.532721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:20.564984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:20.603869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 513/1025] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:20.628256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.631553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b538677-ae12-47ec-9ddb-e48edfa37c6a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.667345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.668040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.668421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.668706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.669410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:20.670263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.671628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:20.676700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 513/1026] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:20.682134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.725543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:20.731106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcbe5e7-0933-424a-9c90-751d83af2379 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.768420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.769366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.769719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.769994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.780327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:20.788570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:20.789665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:20.790691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:20.795102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 514/1027] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:20.795624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581979b7-21b1-4c7c-a2e8-e8c2178ed5d1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.796396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.797802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:20.833150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.833859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.834240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.834524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.835229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] RBAC: Adding query filter params (type=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:20.835539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.836939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:20.841936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 514/1028] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:20.847457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.902584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f191f-a787-4fa3-9071-656c98b7909d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.938710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.939552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.939933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.940313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.941158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:20.941568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.943030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:20.958036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 515/1029] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:20.958225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a1e54-3d75-41cd-b4bf-b1c311d05b98 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:20.959066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:20.979521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:20.993373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:20.994019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:20.996475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:20.996475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:20.996475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] RBAC: Adding query filter params (interface=public, service_id=e5f3e3e9054b47cebacd322acaed4fc3, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:20.996475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:20.997116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.002232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 515/1030] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/endpoints?service_id=e5f3e3e9054b47cebacd322acaed4fc3&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:21.007812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:21.039940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:21.065774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a2b23-95a1-4084-a21a-f26eebe1d5e0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:21.099762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:21.100551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:21.100832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:21.101100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:21.112254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:21.112502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c6cb-dbbc-4a22-bd1e-8ac085de9fd7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:21.113698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:21.130516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:21.131440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:21.137487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:21.137487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:21.137487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:21.137487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:21.137487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 516/1031] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:21.137487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:21.153617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:21.154732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:21.155015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:21.155291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:21.156110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:21.157612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.175695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 516/1032] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:25:21 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:25:21.223015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:21.266521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fbe00f-5629-49b4-aceb-c47d4df81cbe None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:21.303157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:21.304021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:21.304252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:21.304586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:21.305353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:21.305702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:21.307222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.325600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 517/1033] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:21.327619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:21.429772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:21.437167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0052076-10c6-48f3-9c29-8f38df5cd8d4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:21.470248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:21.471307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:21.471822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:21.472270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:21.480276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:21.482569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:21.498861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:21.499945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:25:21.500860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:21.501819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 517/1034] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:25:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:21.507039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:21.608569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5a2d8-6e52-4b15-a89f-602e78d23e1b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:21.636832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:21.642768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:21.643463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:21.643952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:21.644229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:21.645101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:21.645419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:21.646876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.670232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 518/1035] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:25:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:21.671430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:21.769616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18373694-f793-4f1e-8bcc-e30295bec2c8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:21.802969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:21.803607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:21.803915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:21.804188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:21.826906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:21.869354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:21.870782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:21.971910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 518/1036] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:21 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 20:25:21.973218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:22.071681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088d37a-70fc-4660-9393-cbd7ca632fdd None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:22.105073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:22.105852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.106131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.106402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.116798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:22.118230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:22.135842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:22.136756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:22.145554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:22.145554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:22.145554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:22.145554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:22.145554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 519/1037] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:22.145856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:22.198325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:22.249150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05050ccd-7653-4e93-bb97-690b1c57ffd1 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:22.284991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:22.285692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.285979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.286258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.287027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:22.287339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:22.288827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:22.307185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 519/1038] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:22.309011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:22.354225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:22.412500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a4a903-2b81-4c5c-bd48-f8984a7bf4a2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:22.447379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:22.448202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.448482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.448754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.455438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:22.456928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:22.470690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:22.471543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:22.472459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:22.475968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 520/1039] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:22.477766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:22.575980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2af4e9-56eb-4a81-867c-0517bdeb5f05 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:22.589156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:22.609596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:22.610330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.610673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.610951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.611898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:22.612212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:22.613632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aec46-2b65-45eb-8d8f-00b6416857f4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:22.626794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 520/1040] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:25:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:22.632811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:22.633275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039611-a0a0-427b-a1c8-ce5f0bd5824e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.633563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039611-a0a0-427b-a1c8-ce5f0bd5824e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.633839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039611-a0a0-427b-a1c8-ce5f0bd5824e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.635216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 521/1041] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:22.640212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.640495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.640764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.657602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:22.659016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:22.732619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 522/1042] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:22.738799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:22.741763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbef6bf-5995-43f1-8f76-83f82bf74f42 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:22.753653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:22.774911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:22.775558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.775866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.776142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.776955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] RBAC: Adding query filter params (role.id=8d9ef90e4b434f55ba2f803850f15085, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=f310f6d84a3d42df9d473677ac5b6557) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:22.777263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:22.778682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:22.785003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 521/1043] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:22 2022] GET /identity/v3/role_assignments?user.id=f310f6d84a3d42df9d473677ac5b6557&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=8d9ef90e4b434f55ba2f803850f15085 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:22.864277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fa1475-c510-402f-a2ff-e8ee93ccb28d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:22.867137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:22.902690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a0c85-81da-4474-875b-4fa2b32ad7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:22.904852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a0c85-81da-4474-875b-4fa2b32ad7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:22.905350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a0c85-81da-4474-875b-4fa2b32ad7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:22.905827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a0c85-81da-4474-875b-4fa2b32ad7f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:22.907285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a0c85-81da-4474-875b-4fa2b32ad7f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:22.909230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a0c85-81da-4474-875b-4fa2b32ad7f4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:22.911110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 523/1044] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:22.916921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:23.025497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86adf26-b595-4376-97cd-062ca66c71a0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:23.059891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074ae25-8c20-4e8e-8a1c-0210f8a44a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:23.061181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074ae25-8c20-4e8e-8a1c-0210f8a44a0e None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:23.061469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074ae25-8c20-4e8e-8a1c-0210f8a44a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:23.061746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074ae25-8c20-4e8e-8a1c-0210f8a44a0e None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:23.062461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074ae25-8c20-4e8e-8a1c-0210f8a44a0e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:23.063935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074ae25-8c20-4e8e-8a1c-0210f8a44a0e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:23.161979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 522/1045] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:25:22 2022] POST /identity/v3/users => generated 285 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:23.421979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622877d-b25b-4b1e-96fd-1174aa17fb4c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:23.422192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622877d-b25b-4b1e-96fd-1174aa17fb4c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:23.422467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7622877d-b25b-4b1e-96fd-1174aa17fb4c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:23.424122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 524/1046] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:23.429325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:23.429621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:23.429905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:23.446796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:23.448021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:23.462872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:23.515039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 523/1047] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:23.538106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622877d-b25b-4b1e-96fd-1174aa17fb4c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:23.639579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7622877d-b25b-4b1e-96fd-1174aa17fb4c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:23.673388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:23.674089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:23.674369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:23.674661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:23.675325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:23.676702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:23.688132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:23.689144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 06 20:25:23.690003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:23.691757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 525/1048] 10.222.0.41 () {56 vars in 1105 bytes} [Mon Jun 6 20:25:23 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:23.695403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:23.793572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125851a5-b1fa-4459-9cbb-69a4a870135d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:23.826630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:23.827290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:23.827573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:23.827880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:23.828578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] RBAC: Adding query filter params (name=image) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:23.828883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:23.830243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:23.835280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 524/1049] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:23.840673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:23.875609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:23.943479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af237780-e1f9-494a-b69c-506d2abc3de8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:23.977523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:23.978183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:23.978465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:23.978741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:23.979441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:23.979771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:23.981131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e47624-2402-4922-ac0e-48616bfd96b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:23.986275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 526/1050] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:23.991685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:24.007727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:24.091501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf346d3-c6d2-474c-b82e-c344cfa96513 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:24.124186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d2ad1-8346-4539-bf07-d035c3dd57f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:24.124837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2ad1-8346-4539-bf07-d035c3dd57f7 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:24.125118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2ad1-8346-4539-bf07-d035c3dd57f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:24.125392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d2ad1-8346-4539-bf07-d035c3dd57f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:24.126131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d2ad1-8346-4539-bf07-d035c3dd57f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:24.127494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d2ad1-8346-4539-bf07-d035c3dd57f7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:24.207090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 525/1051] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:25:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:25.021172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe32404-7b10-45ab-966e-85256ed85bff None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.021700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe32404-7b10-45ab-966e-85256ed85bff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.021700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe32404-7b10-45ab-966e-85256ed85bff None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.023199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 527/1052] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:25.028354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.028646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.029452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.046839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:25.048046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:25.110154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:25.122308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 526/1053] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:25.146069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe32404-7b10-45ab-966e-85256ed85bff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.248101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:25.252425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe32404-7b10-45ab-966e-85256ed85bff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.287352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f35d10-43df-4854-b4bc-f1dafecec0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.288533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f35d10-43df-4854-b4bc-f1dafecec0aa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.288813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f35d10-43df-4854-b4bc-f1dafecec0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.289080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f35d10-43df-4854-b4bc-f1dafecec0aa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.299291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f35d10-43df-4854-b4bc-f1dafecec0aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-28f35d10-43df-4854-b4bc-f1dafecec0aa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:25.307240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:25.308207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:25.309130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:25.312009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 528/1054] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:25.314494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.416841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb534997-c331-4dfe-b2b0-e49a6a6d074f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.449917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.450574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.451383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.451677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.452417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:25.452718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:25.454080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:25.457551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe78877-c66b-40e5-bec9-eb632d8ff3c2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.457838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe78877-c66b-40e5-bec9-eb632d8ff3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.458118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe78877-c66b-40e5-bec9-eb632d8ff3c2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.459481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 529/1055] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:25.464708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.464995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.465268 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.467163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 527/1056] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:25.472679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.483951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:25.485132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:25.534059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:25.556060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 530/1057] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:25.576657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac36f9-5141-4c48-a0ad-5b595f7794c8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.580504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.611170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.611987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.612269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.612711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.624161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:25.625937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:25.643364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:25.644301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:25.645175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:25.645175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:25.645175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:25.645175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:25.645175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 528/1058] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:25.650123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.680032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:25.690115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9a8f4-9727-4ba1-933f-90daeb70ddd3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.725975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.726770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.727054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.727329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.737608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:25.745178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:25.746095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:25.748060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 531/1059] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:25.752782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.765162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63f52-7c92-444f-8c11-a81467c6f9c4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.776123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:25.799339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.800247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.800531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.800816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.801525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:25.801829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:25.803188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:25.820317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 529/1060] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:25.821007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.861046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a217df7-076f-4f64-a0d5-b3086140559b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.895535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.896232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.896515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.896790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.897758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:25.897890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:25.899205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:25.908392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 532/1061] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:25.914016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:25.924347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b919002-c8fe-45ed-8938-a55dc13854d0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:25.929989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:25.959637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:25.960447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:25.960737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:25.961009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:25.968556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:25.970019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:25.984935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:25.985911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:25.986986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 530/1062] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:25.991677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.020559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce01c9-a973-4f66-9402-c382978b5114 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.022917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.057263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.058127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.058403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.058673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] PATH_INFO: `/v3/users/4baba920903a4251af1a494364f95447` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.072960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] RBAC: Authorizing `identity:get_user(user_id=4baba920903a4251af1a494364f95447)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.074328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.076051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 533/1063] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/users/4baba920903a4251af1a494364f95447 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:26.081739 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.102011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d98002-c061-424e-afb8-6aa08a06ce17 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.139620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.140328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.140619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.140895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.141748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:26.142062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.143553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.157250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.163102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 531/1064] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:25:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:26.168643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.191129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b73461-3332-420a-9c30-8e3869592f9e None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.227971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.228733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.229015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.229287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.269421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.282551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee74955-3f6d-4930-b53a-aaf3e3652b88 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.301105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.302570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.321098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.321886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.322165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.322435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.332897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.334266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:26.353620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:26.354587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:26.355480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:26.355480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:26.355480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:26.355480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:26.355480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 532/1065] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:26.360922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.377115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.409240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 534/1066] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 20:25:26.414862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.472802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817df0e8-a502-457b-9293-bc41899ce355 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.506609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.507268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.507552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.507855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.508573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:26.508884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.510259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fafaf93-bd55-45f1-8017-680dd71dd7b5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.514701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c17ad-07f0-431a-b1e1-a81c8f491848 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.518258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.521707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 533/1067] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:26.522764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94164c9-71fa-46fe-b442-669d25ca7408 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.523081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94164c9-71fa-46fe-b442-669d25ca7408 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.523359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94164c9-71fa-46fe-b442-669d25ca7408 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.524607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 534/1068] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:26.527311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94164c9-71fa-46fe-b442-669d25ca7408 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.554111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d9a0e1-c3fc-4215-8402-9a2c0763b25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.554960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9a0e1-c3fc-4215-8402-9a2c0763b25f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.555242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9a0e1-c3fc-4215-8402-9a2c0763b25f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.555516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d9a0e1-c3fc-4215-8402-9a2c0763b25f None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.556705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9a0e1-c3fc-4215-8402-9a2c0763b25f None admin] RBAC: Authorizing `identity:get_project(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.558130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d9a0e1-c3fc-4215-8402-9a2c0763b25f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.559924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 535/1069] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:26.561061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.561343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.561616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.579163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:26.580386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:26.609106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.631858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94164c9-71fa-46fe-b442-669d25ca7408 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.659690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 536/1070] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:26.659690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.669539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.670327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.670610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.670885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.677512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.678962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:26.693067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:26.694125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 535/1071] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:26.695224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.746951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8baded5-c7ff-431f-8b28-2713749ee54f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.752317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.780460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.781105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.781387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.781663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.782488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a, user.id=4baba920903a4251af1a494364f95447) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:26.783059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.784503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.789784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 537/1072] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/role_assignments?user.id=4baba920903a4251af1a494364f95447&scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:26.790614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.791079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a733ea3-78ab-4b53-aaec-79a631cd573c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.824887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:26.830040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.830707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.830997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.831276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.906585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a083725a-4581-4615-86f6-9c4184033ca6 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:26.906804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.908505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.944911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:26.945573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:26.945851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:26.946130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:26.946922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:26.947233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:26.948723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:26.962289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 538/1073] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:26.969144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:26.983234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:27.029105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 536/1074] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:25:27.035149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:27.072395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:27.081603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4825d6-bb11-4eb9-89f7-79f7c9e7e22f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:27.116158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:27.117040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:27.117325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:27.117672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/f310f6d84a3d42df9d473677ac5b6557/roles/8d9ef90e4b434f55ba2f803850f15085` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:27.119925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=f310f6d84a3d42df9d473677ac5b6557, role_id=8d9ef90e4b434f55ba2f803850f15085)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:27.121363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:27.139487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d54c694-2b86-4460-86b0-561942147296 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:27.173248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:27.173962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:27.174245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:27.174518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:27.186157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:27.187645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:27.190366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 539/1075] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:25:26 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/f310f6d84a3d42df9d473677ac5b6557/roles/8d9ef90e4b434f55ba2f803850f15085 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-99b5048d-e419-494c-aacd-90420e3b3c13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:27.206978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:27.208237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:27.214629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:27.214629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:27.214629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:27.214629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:27.214629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 537/1076] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:27.216816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:27.334161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c616e9-e4b5-454b-9d14-6a0de3980058 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:27.372322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:27.373586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:27.373866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:27.374133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:27.374856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:27.375160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:27.376626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2a88b1-3525-4f96-94b0-1e186f578cac None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:27.391946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 540/1077] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:29.014533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203966-0430-4992-94a6-127d67061366 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.015119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203966-0430-4992-94a6-127d67061366 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.015119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0203966-0430-4992-94a6-127d67061366 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.016472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 538/1078] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:29.021770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.022193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.022293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.038844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:29.040046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:29.047504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:29.105250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 541/1079] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:29.128174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203966-0430-4992-94a6-127d67061366 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.144944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:29.232217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0203966-0430-4992-94a6-127d67061366 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.266050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.266767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.267055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.267334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.277624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:29.285556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:29.286630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:29.288075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 539/1080] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:29.292787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.388901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d9865-18f1-4a6b-ad8d-2ae08505f291 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.420536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.421187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.421465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.421736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.422465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:29.422765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:29.424151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fece7-2454-45e1-a4ef-8f4b6a5f2020 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:29.432358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 542/1081] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:29.438746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.469212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7189756-a0ec-4c8a-8b55-20d284cb877b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.469493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7189756-a0ec-4c8a-8b55-20d284cb877b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.469762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7189756-a0ec-4c8a-8b55-20d284cb877b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.471093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 543/1082] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:25:29.476040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.476323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.476598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.482429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:29.493658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:29.499775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:29.507865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:29.536906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7903fc5d-6521-46fe-8038-6c632e1c62b1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.562252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 544/1083] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:25:29.570285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.571141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.571425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.571724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] PATH_INFO: `/v3/users/4baba920903a4251af1a494364f95447` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.572914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] RBAC: Authorizing `identity:get_user(user_id=4baba920903a4251af1a494364f95447)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:29.574323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:29.581160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 540/1084] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/users/4baba920903a4251af1a494364f95447 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:29.582229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.587269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.679364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a88025-3d5b-4910-b8f3-b646ea26a32c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.683596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150469ff-0fc5-4708-b2ff-169803e0e816 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.712590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.713429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.713707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.713978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.715127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] RBAC: Authorizing `identity:get_project(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:29.716627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:29.718294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 545/1085] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:29.720905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.721779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.722134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.722481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.724049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.726538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:29.730322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:29.733800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:29.741899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:29.742809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:29.743652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 541/1086] 10.222.0.41 () {56 vars in 1120 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:29.749433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.821962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2e533-f796-4ab5-99c1-0d3105db451c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.846424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa933a2-133a-469f-9934-f295bc5e069a None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:29.855137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.856050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.856337 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.856613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a/users/4baba920903a4251af1a494364f95447/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.859446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a, user_id=4baba920903a4251af1a494364f95447, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:29.860913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:29.880797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:29.882781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:29.882781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:29.882781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:29.882781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:29.883120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:29.884352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:29.892007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 542/1087] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:29.897527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:29.937417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 546/1088] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:25:29 2022] PUT /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a/users/4baba920903a4251af1a494364f95447/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:25:29.957767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:30.000478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad600e-26bc-4464-bac9-193d6859c14d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:30.037301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:30.038114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:30.038402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:30.038684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:30.051222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:30.052718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:30.070500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:30.071697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:30.073273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:30.073273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:30.073273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:30.073273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:30.073273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 543/1089] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 06 20:25:30.077606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:30.142000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:30.180610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6cfc3d-d76f-4e55-801c-2080a252c3b4 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:30.214933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:30.215591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:30.215907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:30.216201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:30.216924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:30.217236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:30.218597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:30.229940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 547/1090] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:30.235630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:30.334583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c426db-b7a2-4e6a-8c86-e368d6df77ad None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:30.347774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:30.367507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:30.368316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:30.368602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:30.368882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:30.377056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:30.378493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:25:30.392107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:30.393224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:30.394611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 544/1091] 10.222.0.41 () {56 vars in 1093 bytes} [Mon Jun 6 20:25:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:30.398980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:30.496108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b572c3-7b5a-4931-b664-51e9e5ac1268 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:30.518000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:30.529170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:30.529834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:30.530158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:30.530435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:30.531256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:30.531579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:30.533322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:30.551213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 548/1092] 10.222.0.41 () {56 vars in 1133 bytes} [Mon Jun 6 20:25:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:30.556962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:30.653790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50628d-8162-45c7-9ce2-d2f6c9399182 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:30.687710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:30.688353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:30.688640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:30.688916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:30.729655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:30.754238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:30.755989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:30.850228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 545/1093] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:25:30.855871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:30.950190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf712f-5cfb-4627-a021-741b6aa810bc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:30.982723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:30.983491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:30.983799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:30.984068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:30.993599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:30.995019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:25:31.011677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:31.012702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:31.013624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:25:31.013624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:25:31.013624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:25:31.013624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:31.013624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 549/1094] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:31.018559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:31.092663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:31.117149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4cf37f-ffec-4129-bac9-9b8e9567b840 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:31.150133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:31.150791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:31.151074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:31.151351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:31.152096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:31.152403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:31.154374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:31.165961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 546/1095] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:31.171309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:31.201777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:31.269580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6181cc8-5209-4530-b06a-8d0a24f9f991 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:31.302590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:31.303367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:31.303657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:31.303956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:31.310387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:31.311851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:25:31.325288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:31.326857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:25:31.327984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:31.331888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 550/1096] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:25:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:31.332734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:31.429042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e591998-95e3-42b1-803c-45138ab0e9a3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:31.461423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:31.462009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:31.462292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:31.462565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:31.463332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:31.463639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:31.465085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c6170d-746e-4b1a-974d-eacb3122d74e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:31.467249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:31.477795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 547/1097] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:25:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:31.483792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:31.573836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:31.582704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385b26a-bea1-45a2-87e8-c5578f737ab8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:31.620995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:31.621656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:31.621942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:31.622219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:31.623575 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] RBAC: Adding query filter params (role.id=8d9ef90e4b434f55ba2f803850f15085, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=f310f6d84a3d42df9d473677ac5b6557) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:31.623919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:31.625365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c42e44-fbf6-479f-90f6-a399caa345f8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:31.631374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 551/1098] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:31 2022] GET /identity/v3/role_assignments?user.id=f310f6d84a3d42df9d473677ac5b6557&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=8d9ef90e4b434f55ba2f803850f15085 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:32.155114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2319f-2ba1-4c40-a63f-52d3dcb2fa22 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:32.155646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2319f-2ba1-4c40-a63f-52d3dcb2fa22 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:32.155781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e2319f-2ba1-4c40-a63f-52d3dcb2fa22 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:32.157057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 548/1099] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:32.162694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:32.163172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:32.163454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:32.180773 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:32.181961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:32.254770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 552/1100] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:32.278164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2319f-2ba1-4c40-a63f-52d3dcb2fa22 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:32.381113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e2319f-2ba1-4c40-a63f-52d3dcb2fa22 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:32.414820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:32.415518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:32.415826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:32.416099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:32.426629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:32.434171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:32.435139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:32.436059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 549/1101] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:25:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:32.441616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:32.527624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:32.542482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7527-64ee-4b75-9849-6ca07b22f76a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:32.575106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:32.575812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:32.576099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:32.576378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:32.577116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:32.577424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:32.578781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:32.586511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 553/1102] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:32.592784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:32.645944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:32.688571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832a071-4d1b-4519-9cac-c3b605e695da None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:32.720359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:32.721196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:32.721474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:32.721743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] PATH_INFO: `/v3/users/4baba920903a4251af1a494364f95447` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:32.722889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] RBAC: Authorizing `identity:get_user(user_id=4baba920903a4251af1a494364f95447)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:32.724338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:32.725919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 550/1103] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:32 2022] GET /identity/v3/users/4baba920903a4251af1a494364f95447 => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:32.731516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:32.825332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caff33b-0dbd-4a5e-bb0a-b6a1bfba4c63 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:32.857722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:32.858345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:32.858627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:32.858899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:32.875358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:32.923015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:32.924473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:33.017324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 554/1104] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:25:33.022991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:33.119944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d8d2e-b91c-4510-b22c-cbc2ff4e7862 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:33.141620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:33.152469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e74ff9b-a4dc-48c4-a231-f8dc23012e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:33.154770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74ff9b-a4dc-48c4-a231-f8dc23012e7b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:33.154770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74ff9b-a4dc-48c4-a231-f8dc23012e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:33.154770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e74ff9b-a4dc-48c4-a231-f8dc23012e7b None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:33.155098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74ff9b-a4dc-48c4-a231-f8dc23012e7b None admin] RBAC: Authorizing `identity:get_project(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:33.156443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e74ff9b-a4dc-48c4-a231-f8dc23012e7b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:33.158358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 551/1105] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:33 2022] GET /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:33.164287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:33.211378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:33.260008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b11568-16cf-4a38-bc10-ad1e8425ec56 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:33.293079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:33.293665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:33.293947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:33.294225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:33.295037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a, user.id=4baba920903a4251af1a494364f95447) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:33.295392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:33.296884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e75536-3f07-4cbd-9072-c21c168d7dce None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:33.302596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 555/1106] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:33 2022] GET /identity/v3/role_assignments?user.id=4baba920903a4251af1a494364f95447&scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:35.531558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33c59-937f-454c-94fe-b4a9dcb1f2dd None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:35.532087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33c59-937f-454c-94fe-b4a9dcb1f2dd None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:35.532087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba33c59-937f-454c-94fe-b4a9dcb1f2dd None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:35.533682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 552/1107] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:35.538984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:35.539395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:35.539525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:35.556924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:35.558082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:35.620748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 556/1108] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:35.650926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33c59-937f-454c-94fe-b4a9dcb1f2dd None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:35.748181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba33c59-937f-454c-94fe-b4a9dcb1f2dd None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:35.780354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144c5dc-9e58-4ee4-8a7f-461d6b70c2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:35.781115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144c5dc-9e58-4ee4-8a7f-461d6b70c2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:35.781395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144c5dc-9e58-4ee4-8a7f-461d6b70c2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:35.781716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144c5dc-9e58-4ee4-8a7f-461d6b70c2a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:35.783068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144c5dc-9e58-4ee4-8a7f-461d6b70c2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:35.784526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144c5dc-9e58-4ee4-8a7f-461d6b70c2a4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:35.786108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 553/1109] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:35.791634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:35.832308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:35.884415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad7b67-0ca1-4e4c-9625-52ee8b1629bb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:35.917366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82874ea-99e5-4d85-aed2-a1a7279975a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:35.918011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82874ea-99e5-4d85-aed2-a1a7279975a8 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:35.918291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82874ea-99e5-4d85-aed2-a1a7279975a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:35.918564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82874ea-99e5-4d85-aed2-a1a7279975a8 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:35.919295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82874ea-99e5-4d85-aed2-a1a7279975a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:35.920738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82874ea-99e5-4d85-aed2-a1a7279975a8 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:35.993894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 557/1110] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:25:35 2022] POST /identity/v3/users => generated 286 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:38.231340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9715d-0893-4ef0-86b4-f0eed9638aa3 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:38.232358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9715d-0893-4ef0-86b4-f0eed9638aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:38.232544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9715d-0893-4ef0-86b4-f0eed9638aa3 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:38.233931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 554/1111] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:38.239397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:38.239830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:38.239975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:38.256050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:38.257214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:38.321149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 558/1112] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:38.344834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9715d-0893-4ef0-86b4-f0eed9638aa3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:38.403045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:38.444209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9715d-0893-4ef0-86b4-f0eed9638aa3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:38.476413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:38.477250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:38.477535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:38.477810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] PATH_INFO: `/v3/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:38.479315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] RBAC: Authorizing `identity:get_role(role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:38.480667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:38.482064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 555/1113] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:38 2022] GET /identity/v3/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:38.487819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:38.514872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:38.583424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd0116-3cbb-4d11-979b-241815b07bca None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:38.615576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:38.616448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:38.616731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:38.617003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] PATH_INFO: `/v3/users/f309dbae04384706b5b6df74f86d88df` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:38.630023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] RBAC: Authorizing `identity:get_user(user_id=f309dbae04384706b5b6df74f86d88df)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:38.631448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:38.633064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 559/1114] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:38 2022] GET /identity/v3/users/f309dbae04384706b5b6df74f86d88df => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:38.638654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:38.733953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62c6535-7ba3-47e9-acb5-28dd4bca0e61 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:38.758016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:38.766320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:38.766948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:38.773579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:38.773579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:38.832075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:38.833501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:38.926811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 556/1115] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:25:38.932598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:39.013243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:39.026659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4cf31-9c40-414e-93a4-27b0900832fa None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:39.059548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f28699-bfab-459e-b3be-4fe707ec7c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:39.060429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f28699-bfab-459e-b3be-4fe707ec7c6b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:39.060706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f28699-bfab-459e-b3be-4fe707ec7c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:39.060976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f28699-bfab-459e-b3be-4fe707ec7c6b None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:39.062129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f28699-bfab-459e-b3be-4fe707ec7c6b None admin] RBAC: Authorizing `identity:get_project(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:39.063549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f28699-bfab-459e-b3be-4fe707ec7c6b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:39.065215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 560/1116] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:38 2022] GET /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:39.071079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:39.098558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:39.168236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91762b-075e-4071-9f19-b67dbd0a81ed None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:39.200500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:39.201132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:39.201413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:39.201687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:39.202500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] RBAC: Adding query filter params (role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a, user.id=f309dbae04384706b5b6df74f86d88df) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:39.202809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:39.204243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cee9f5d-3a52-4543-ba76-f4f45f2a8e34 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:39.209414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 557/1117] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:39 2022] GET /identity/v3/role_assignments?user.id=f309dbae04384706b5b6df74f86d88df&scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:41.388683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d652e-65f9-4515-999e-6337fd05020e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:41.389194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d652e-65f9-4515-999e-6337fd05020e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:41.389194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d652e-65f9-4515-999e-6337fd05020e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:41.390554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 561/1118] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:41.395613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:41.395999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:41.396213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:41.413373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:41.414550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:41.477958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 558/1119] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:41.500661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d652e-65f9-4515-999e-6337fd05020e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:41.596539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d652e-65f9-4515-999e-6337fd05020e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:41.629613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:41.630441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:41.630718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:41.630986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] PATH_INFO: `/v3/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:41.632449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] RBAC: Authorizing `identity:get_role(role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:41.633807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:41.635107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 562/1120] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:41 2022] GET /identity/v3/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:41.640637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:41.660692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:41.734699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf5698-2ade-4160-8977-ce924a3e75e5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:41.767415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:41.768237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:41.768521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:41.768798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] PATH_INFO: `/v3/users/f309dbae04384706b5b6df74f86d88df` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:41.769957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] RBAC: Authorizing `identity:get_user(user_id=f309dbae04384706b5b6df74f86d88df)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:41.771387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:41.773017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 559/1121] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:41 2022] GET /identity/v3/users/f309dbae04384706b5b6df74f86d88df => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:41.778723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:41.802819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:41.875311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a230beb-f01e-4d05-8272-2c8f2497c61d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:41.907244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a09b75a-10f7-48b5-a98e-28654b98b7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:41.908365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b75a-10f7-48b5-a98e-28654b98b7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:41.908648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b75a-10f7-48b5-a98e-28654b98b7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:41.908926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a09b75a-10f7-48b5-a98e-28654b98b7c9 None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:41.910078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09b75a-10f7-48b5-a98e-28654b98b7c9 None admin] RBAC: Authorizing `identity:get_project(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:41.911495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a09b75a-10f7-48b5-a98e-28654b98b7c9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:41.913170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 563/1122] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:41 2022] GET /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:41.919404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:42.003417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:42.014659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a204106-e77b-45d0-886a-15f387e031a3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:42.047584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700f050-84b9-4cc2-9c31-3e636cc254b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:42.048420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700f050-84b9-4cc2-9c31-3e636cc254b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:42.048701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700f050-84b9-4cc2-9c31-3e636cc254b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:42.048975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700f050-84b9-4cc2-9c31-3e636cc254b7 None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a/users/f309dbae04384706b5b6df74f86d88df/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:42.051098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700f050-84b9-4cc2-9c31-3e636cc254b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a, user_id=f309dbae04384706b5b6df74f86d88df, role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:42.052534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700f050-84b9-4cc2-9c31-3e636cc254b7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:42.136376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 560/1123] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:25:41 2022] PUT /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a/users/f309dbae04384706b5b6df74f86d88df/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:25:44.316762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21ce3-909b-42bb-a52a-34ce38dd0a31 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:44.317262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21ce3-909b-42bb-a52a-34ce38dd0a31 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:44.317372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c21ce3-909b-42bb-a52a-34ce38dd0a31 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:44.318647 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 564/1124] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:44.323869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:44.324240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:44.324429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:44.340129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:44.341815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:44.405246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 561/1125] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:44.428872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c21ce3-909b-42bb-a52a-34ce38dd0a31 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:44.514477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:44.524151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c21ce3-909b-42bb-a52a-34ce38dd0a31 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:44.557066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:44.557893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:44.558169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:44.558436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] PATH_INFO: `/v3/roles/dfe72d3705244e3c8f4a955aea7b170b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:44.559920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] RBAC: Authorizing `identity:get_role(role_id=dfe72d3705244e3c8f4a955aea7b170b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:44.561263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:44.562571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 565/1126] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:44 2022] GET /identity/v3/roles/dfe72d3705244e3c8f4a955aea7b170b => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:44.568187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:44.641686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:44.662243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1abc91c-65d5-4cc0-9930-d146dbf544f1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:44.694246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:44.695089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:44.695368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:44.695638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] PATH_INFO: `/v3/users/f309dbae04384706b5b6df74f86d88df` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:44.696816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] RBAC: Authorizing `identity:get_user(user_id=f309dbae04384706b5b6df74f86d88df)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:44.698222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:44.699831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 562/1127] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:44 2022] GET /identity/v3/users/f309dbae04384706b5b6df74f86d88df => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:44.705855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:44.801307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03133e61-4803-42f8-9010-0078343d3754 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:44.833633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:44.834266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:44.834548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:44.834823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:44.860533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:44.898937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:44.900412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:44.993183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 566/1128] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:25:44.999419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:45.094675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200411-8202-4a91-b30a-345060bece74 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:45.126599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3fca-3cb3-4f73-abef-e97f2d83b40e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:45.127422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3fca-3cb3-4f73-abef-e97f2d83b40e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:45.127726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3fca-3cb3-4f73-abef-e97f2d83b40e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:45.128003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3fca-3cb3-4f73-abef-e97f2d83b40e None admin] PATH_INFO: `/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:45.129154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3fca-3cb3-4f73-abef-e97f2d83b40e None admin] RBAC: Authorizing `identity:get_project(project_id=cb1060e4b5ff420fa7e69ec17b9abd0a)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:45.130585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3fca-3cb3-4f73-abef-e97f2d83b40e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:45.132253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 563/1129] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:44 2022] GET /identity/v3/projects/cb1060e4b5ff420fa7e69ec17b9abd0a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:45.138584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:45.194939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:45.233589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7fb5ad-d263-4503-a656-69284572c17b None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:45.267099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:45.267777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:45.268063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:45.268338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:45.269152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] RBAC: Adding query filter params (role.id=dfe72d3705244e3c8f4a955aea7b170b, scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a, user.id=f309dbae04384706b5b6df74f86d88df) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:45.269462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:45.270884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed099c3-bf4e-4d32-9fc8-36cfeba734a4 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:45.276504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 567/1130] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:45 2022] GET /identity/v3/role_assignments?user.id=f309dbae04384706b5b6df74f86d88df&scope.project.id=cb1060e4b5ff420fa7e69ec17b9abd0a&role.id=dfe72d3705244e3c8f4a955aea7b170b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:47.497629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a39b-73e3-4eb7-86c6-5ae3898c32f2 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:47.498149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a39b-73e3-4eb7-86c6-5ae3898c32f2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:47.498149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453a39b-73e3-4eb7-86c6-5ae3898c32f2 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:47.499445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 564/1131] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:47.504706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:47.505092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:47.505261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:47.522072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:47.523245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:47.586159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 568/1132] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:47.610215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a39b-73e3-4eb7-86c6-5ae3898c32f2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:47.625675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:47.708190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453a39b-73e3-4eb7-86c6-5ae3898c32f2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:47.743609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247b663-4eaf-418f-a568-954a7ee5a38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:47.744408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247b663-4eaf-418f-a568-954a7ee5a38e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:47.744689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247b663-4eaf-418f-a568-954a7ee5a38e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:47.744960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247b663-4eaf-418f-a568-954a7ee5a38e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:47.746097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247b663-4eaf-418f-a568-954a7ee5a38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:47.747550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247b663-4eaf-418f-a568-954a7ee5a38e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:47.749198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 565/1133] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:47.754978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:47.805758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:47.850067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61a0d4-a474-4af0-9073-29671f51a5c0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:47.881436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd279c-712a-4c14-95b4-d14cd19de559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:47.882154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd279c-712a-4c14-95b4-d14cd19de559 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:47.882432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd279c-712a-4c14-95b4-d14cd19de559 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:47.882701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd279c-712a-4c14-95b4-d14cd19de559 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:47.883395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd279c-712a-4c14-95b4-d14cd19de559 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:47.884824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd279c-712a-4c14-95b4-d14cd19de559 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:47.948848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 569/1134] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:25:47 2022] POST /identity/v3/projects => generated 311 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:25:50.186966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bb236-991e-4d1c-b5b3-5b4dd592aa82 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:50.187496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bb236-991e-4d1c-b5b3-5b4dd592aa82 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:50.187586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571bb236-991e-4d1c-b5b3-5b4dd592aa82 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:50.188924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 566/1135] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:50.194085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:50.194454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:50.194633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:50.211239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:50.212436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:50.276202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 570/1136] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:50.300831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bb236-991e-4d1c-b5b3-5b4dd592aa82 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:50.380640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:50.399072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571bb236-991e-4d1c-b5b3-5b4dd592aa82 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:50.431574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843ff611-ab0a-46c3-9a7e-70c8b9c99c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:50.432383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff611-ab0a-46c3-9a7e-70c8b9c99c32 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:50.432668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff611-ab0a-46c3-9a7e-70c8b9c99c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:50.432947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843ff611-ab0a-46c3-9a7e-70c8b9c99c32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:50.434107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ff611-ab0a-46c3-9a7e-70c8b9c99c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:50.436133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843ff611-ab0a-46c3-9a7e-70c8b9c99c32 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:50.437776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 567/1137] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:25:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:50.443452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:50.475349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:50.538133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07f8e8-3a76-4d8e-8c63-b86512fb0553 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:50.569973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596010e-4b49-402c-a909-b9fc99933d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:50.570612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596010e-4b49-402c-a909-b9fc99933d8d None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:50.570893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596010e-4b49-402c-a909-b9fc99933d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:50.571166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596010e-4b49-402c-a909-b9fc99933d8d None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:50.571886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596010e-4b49-402c-a909-b9fc99933d8d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:50.573288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596010e-4b49-402c-a909-b9fc99933d8d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:50.644859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 571/1138] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:25:50 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:25:52.883967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d0ae6-3a6b-4207-bab4-87059f9b0e56 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:52.884498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d0ae6-3a6b-4207-bab4-87059f9b0e56 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:52.884587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d0ae6-3a6b-4207-bab4-87059f9b0e56 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:52.885889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 568/1139] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:52.891532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:52.891959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:52.892116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:52.909143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:52.910313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:52.973383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 572/1140] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:52.996681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d0ae6-3a6b-4207-bab4-87059f9b0e56 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:53.093155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d0ae6-3a6b-4207-bab4-87059f9b0e56 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:53.124562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:53.125325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:53.125667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:53.125950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:53.136016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:53.144048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:53.145133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:53.145997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 569/1141] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:53.150987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:53.171918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:53.247959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f6832-53b3-4272-a409-64ced5eaa671 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:53.280477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:53.281129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:53.281411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:53.281685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:53.282679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:53.282982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:53.284409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:53.292161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 573/1142] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:53.297929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:53.310947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:53.395073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1eb16-f37e-479e-bb55-6fec9f4934f0 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:53.427300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:53.428177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:53.428464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:53.428740 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] PATH_INFO: `/v3/users/21b66eae4f4e4c58b80a996ba8fb3a08` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:53.442063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] RBAC: Authorizing `identity:get_user(user_id=21b66eae4f4e4c58b80a996ba8fb3a08)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:53.443501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:53.445201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 570/1143] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:53 2022] GET /identity/v3/users/21b66eae4f4e4c58b80a996ba8fb3a08 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:53.451466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:53.529609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:53.546692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fb0146-c443-482a-bc52-43ff11d83b64 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:53.579196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:53.579860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:53.580146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:53.580429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:53.644390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:53.645871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:53.721483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:53.739539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 574/1144] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 06 20:25:53.745352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:53.812527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:53.843054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f97f2-bf5c-4e57-a3a0-8fd342a749d8 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:53.875218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828d881-791f-4f59-96f9-da3d0f51f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:53.876077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d881-791f-4f59-96f9-da3d0f51f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:53.876359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d881-791f-4f59-96f9-da3d0f51f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:53.876631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828d881-791f-4f59-96f9-da3d0f51f4c2 None admin] PATH_INFO: `/v3/projects/7dfe3081517f45558c82147dff35162d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:53.877768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d881-791f-4f59-96f9-da3d0f51f4c2 None admin] RBAC: Authorizing `identity:get_project(project_id=7dfe3081517f45558c82147dff35162d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:53.879179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828d881-791f-4f59-96f9-da3d0f51f4c2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:53.880840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 571/1145] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:53 2022] GET /identity/v3/projects/7dfe3081517f45558c82147dff35162d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:53.886787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:53.981876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b6b1d-2344-465d-b129-1917bcbe595d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:54.013789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:54.014417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:54.014704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:54.014978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:54.015822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=7dfe3081517f45558c82147dff35162d, user.id=21b66eae4f4e4c58b80a996ba8fb3a08) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:54.016135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:54.017539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8087bae-b67b-4cd5-a026-d1aac0c6dae3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:54.022767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 575/1146] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:25:53 2022] GET /identity/v3/role_assignments?user.id=21b66eae4f4e4c58b80a996ba8fb3a08&scope.project.id=7dfe3081517f45558c82147dff35162d&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:56.190958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223468b4-70b8-4098-bc5d-6dc9c4ef9a04 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:56.191492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223468b4-70b8-4098-bc5d-6dc9c4ef9a04 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:56.191492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223468b4-70b8-4098-bc5d-6dc9c4ef9a04 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:56.192884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 572/1147] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:56.198098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:56.198472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:56.198652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:56.215318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:56.217127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:56.257408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:56.280774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 576/1148] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:56.303329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223468b4-70b8-4098-bc5d-6dc9c4ef9a04 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:56.400446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223468b4-70b8-4098-bc5d-6dc9c4ef9a04 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:56.434025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:56.435329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:56.435608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:56.435918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:56.438452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:56.447602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:56.454190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:56.455386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:25:56.456260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 573/1149] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:56.461262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:56.554484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df444906-6605-4495-90a9-dd32b61d25e3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:56.586998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:56.587634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:56.587955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:56.588224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:56.588946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:56.589246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:56.590617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:56.598235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 577/1150] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:56.603867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:56.699139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d266146-3c0a-4dc8-be3f-0cd5df00df70 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:56.730186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:56.731565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:56.731878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:56.732154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] PATH_INFO: `/v3/users/21b66eae4f4e4c58b80a996ba8fb3a08` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:56.733313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] RBAC: Authorizing `identity:get_user(user_id=21b66eae4f4e4c58b80a996ba8fb3a08)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:56.734746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:56.736362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 574/1151] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:56 2022] GET /identity/v3/users/21b66eae4f4e4c58b80a996ba8fb3a08 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:56.742015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:56.780268 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:56.838716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c68c6e-16a1-48ec-b054-9cbee41b432f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:56.870303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee25f7b-5603-46f1-bd2c-7199800f3ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:56.871135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25f7b-5603-46f1-bd2c-7199800f3ccd None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:56.871411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25f7b-5603-46f1-bd2c-7199800f3ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:56.872315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee25f7b-5603-46f1-bd2c-7199800f3ccd None admin] PATH_INFO: `/v3/projects/7dfe3081517f45558c82147dff35162d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:56.873462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee25f7b-5603-46f1-bd2c-7199800f3ccd None admin] RBAC: Authorizing `identity:get_project(project_id=7dfe3081517f45558c82147dff35162d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:56.874883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee25f7b-5603-46f1-bd2c-7199800f3ccd None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:56.876539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 578/1152] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:25:56 2022] GET /identity/v3/projects/7dfe3081517f45558c82147dff35162d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:56.882215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:56.943320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:56.980548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08962b6c-826e-4954-9d24-01af05151215 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:57.012620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f2a1f-4bf8-4d92-80c5-9e4cbb131783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:57.013506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f2a1f-4bf8-4d92-80c5-9e4cbb131783 None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:57.013788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f2a1f-4bf8-4d92-80c5-9e4cbb131783 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:57.014064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f2a1f-4bf8-4d92-80c5-9e4cbb131783 None admin] PATH_INFO: `/v3/projects/7dfe3081517f45558c82147dff35162d/users/21b66eae4f4e4c58b80a996ba8fb3a08/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:57.016053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f2a1f-4bf8-4d92-80c5-9e4cbb131783 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dfe3081517f45558c82147dff35162d, user_id=21b66eae4f4e4c58b80a996ba8fb3a08, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:57.017473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f2a1f-4bf8-4d92-80c5-9e4cbb131783 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:57.091436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 575/1153] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:25:56 2022] PUT /identity/v3/projects/7dfe3081517f45558c82147dff35162d/users/21b66eae4f4e4c58b80a996ba8fb3a08/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:25:59.301152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184b8d5-e8c3-4275-9a43-494808489980 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:59.301700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184b8d5-e8c3-4275-9a43-494808489980 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:59.301799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2184b8d5-e8c3-4275-9a43-494808489980 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:59.303130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 579/1154] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:25:59.308193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:59.308614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:59.308741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:59.324928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:25:59.326114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:25:59.390449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 576/1155] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:25:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:25:59.413461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184b8d5-e8c3-4275-9a43-494808489980 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:59.509884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2184b8d5-e8c3-4275-9a43-494808489980 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:59.516896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:59.542484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:59.543244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:59.543549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:59.543847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:59.554006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:25:59.561977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:25:59.562937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:25:59.563868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 580/1156] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:25:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:25:59.569005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:59.621170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:59.662698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732614e-ee76-4c0b-8555-54394487fc6f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:59.695590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:59.696261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:59.696546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:59.696821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:59.697557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:25:59.697861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:59.699222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:59.706923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 577/1157] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:25:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:25:59.712406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:59.807645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479d7f8d-8d7f-4b75-be28-7ef52004b3bb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:59.839772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:59.840644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:59.840927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:59.841200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] PATH_INFO: `/v3/users/21b66eae4f4e4c58b80a996ba8fb3a08` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:25:59.842359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=21b66eae4f4e4c58b80a996ba8fb3a08)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:25:59.843809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:25:59.845396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 581/1158] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:25:59 2022] GET /identity/v3/users/21b66eae4f4e4c58b80a996ba8fb3a08 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:25:59.851204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:25:59.946803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55f151-3134-4eae-a746-885306cf985e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:25:59.961712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:25:59.978659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:25:59.979513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:25:59.979794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:25:59.979880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:00.044110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:00.045553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:00.138526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 578/1159] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:25:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:26:00.144181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:00.178033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:00.240808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c0ebd-72b6-4c27-a875-0143cda2b9d7 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:00.273452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5910de4-22ff-4d05-bbcd-19aedea23504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:00.274298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5910de4-22ff-4d05-bbcd-19aedea23504 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:00.274580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5910de4-22ff-4d05-bbcd-19aedea23504 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:00.274854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5910de4-22ff-4d05-bbcd-19aedea23504 None admin] PATH_INFO: `/v3/projects/7dfe3081517f45558c82147dff35162d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:00.276029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5910de4-22ff-4d05-bbcd-19aedea23504 None admin] RBAC: Authorizing `identity:get_project(project_id=7dfe3081517f45558c82147dff35162d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:00.277440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5910de4-22ff-4d05-bbcd-19aedea23504 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:00.279116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 582/1160] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:26:00 2022] GET /identity/v3/projects/7dfe3081517f45558c82147dff35162d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:00.285344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:00.302017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:00.379901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892b48-ddcc-4c4b-859b-989156b16df2 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:00.411557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:00.412166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:00.412452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:00.412727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:00.413535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=7dfe3081517f45558c82147dff35162d, user.id=21b66eae4f4e4c58b80a996ba8fb3a08) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:00.413847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:00.415260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecd0761-f378-4d5e-be20-829d309e7853 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:00.421440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 579/1161] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:26:00 2022] GET /identity/v3/role_assignments?user.id=21b66eae4f4e4c58b80a996ba8fb3a08&scope.project.id=7dfe3081517f45558c82147dff35162d&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:02.621451 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627f3e-fc8a-4079-918c-38f6a4733b77 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:02.621952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627f3e-fc8a-4079-918c-38f6a4733b77 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:02.621952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80627f3e-fc8a-4079-918c-38f6a4733b77 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:02.623292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 583/1162] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:02.628387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:02.628824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:02.628949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:02.645608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:02.646796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:02.710164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 580/1163] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 06 20:26:02.732504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80627f3e-fc8a-4079-918c-38f6a4733b77 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:02.829065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80627f3e-fc8a-4079-918c-38f6a4733b77 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:02.830509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:02.861331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:02.861943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:02.862222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:02.862495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:02.925691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:02.927043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:03.020059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 584/1164] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:26:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:26:03.025910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:03.037265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:03.118644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7bf46-a2f3-46f5-a71f-5dac09e29f70 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:03.151147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:03.151948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:03.152233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:03.152508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:03.159179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:03.160632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-03647eac-6260-4992-9eec-4bf94f44fc66 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:26:03.174342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:26:03.175514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:26:03.176761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 581/1165] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:26:03 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:26:03.181229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:03.183341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:03.276802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9502b9-3517-4f10-bc89-d1eb6ecfcf24 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:03.309545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:03.310183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:03.310460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:03.310732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:03.311434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:03.311757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:03.313112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675e46e-2262-4352-9185-60ce93d5432f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:03.319948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 585/1166] 10.222.0.41 () {56 vars in 1115 bytes} [Mon Jun 6 20:26:03 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:05.497282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174394d9-ab15-41ac-8408-5f8bea8c50ce None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:05.497787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174394d9-ab15-41ac-8408-5f8bea8c50ce None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:05.497787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174394d9-ab15-41ac-8408-5f8bea8c50ce None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:05.499159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 582/1167] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:05.504270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000615c7-3b29-4156-8dce-579d7df8e1d2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:05.504684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000615c7-3b29-4156-8dce-579d7df8e1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:05.504822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000615c7-3b29-4156-8dce-579d7df8e1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:05.520923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-000615c7-3b29-4156-8dce-579d7df8e1d2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:05.522095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-000615c7-3b29-4156-8dce-579d7df8e1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:05.578017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:05.584876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 586/1168] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:26:05.608140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174394d9-ab15-41ac-8408-5f8bea8c50ce None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:05.675881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:05.705796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174394d9-ab15-41ac-8408-5f8bea8c50ce None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:05.737471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315de72-8ace-4ae9-bda6-9f0b19a198bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:05.738109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315de72-8ace-4ae9-bda6-9f0b19a198bf None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:05.738395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315de72-8ace-4ae9-bda6-9f0b19a198bf None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:05.738670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315de72-8ace-4ae9-bda6-9f0b19a198bf None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:05.739405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315de72-8ace-4ae9-bda6-9f0b19a198bf None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:05.740790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315de72-8ace-4ae9-bda6-9f0b19a198bf None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:05.794064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 583/1169] 10.222.0.41 () {60 vars in 1138 bytes} [Mon Jun 6 20:26:05 2022] POST /identity/v3/domains => generated 267 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:26:08.040381 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6972669-4c44-4688-9ed8-982a386a736b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:08.040935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6972669-4c44-4688-9ed8-982a386a736b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:08.040935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6972669-4c44-4688-9ed8-982a386a736b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:08.042283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 587/1170] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:08.047970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:08.048347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:08.048543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:08.065486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:08.066665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:08.130305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 584/1171] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:26:08.153649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6972669-4c44-4688-9ed8-982a386a736b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:08.253046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6972669-4c44-4688-9ed8-982a386a736b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:08.285540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b06753-8a24-4ce2-9e59-9672322d1759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:08.286392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b06753-8a24-4ce2-9e59-9672322d1759 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:08.286676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b06753-8a24-4ce2-9e59-9672322d1759 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:08.286952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b06753-8a24-4ce2-9e59-9672322d1759 None admin] PATH_INFO: `/v3/domains/fe22baa411a9427fb27377880da93aa0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:08.297571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b06753-8a24-4ce2-9e59-9672322d1759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe22baa411a9427fb27377880da93aa0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:08.298954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b06753-8a24-4ce2-9e59-9672322d1759 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:08.300586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 588/1172] 10.222.0.41 () {56 vars in 1183 bytes} [Mon Jun 6 20:26:08 2022] GET /identity/v3/domains/fe22baa411a9427fb27377880da93aa0 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:08.306312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:08.362173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:08.399281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc189c-295f-4bba-9b22-e7009d9d7fa3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:08.432955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b437e-d787-49a7-be97-6585575ea931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:08.433634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b437e-d787-49a7-be97-6585575ea931 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:08.434107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b437e-d787-49a7-be97-6585575ea931 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:08.434809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b437e-d787-49a7-be97-6585575ea931 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:08.435506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b437e-d787-49a7-be97-6585575ea931 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:08.436950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b437e-d787-49a7-be97-6585575ea931 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:08.519385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 585/1173] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:26:08 2022] POST /identity/v3/projects => generated 361 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:26:10.762251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84978428-210c-4c36-b452-d79527d7f655 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:10.762781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84978428-210c-4c36-b452-d79527d7f655 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:10.762781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84978428-210c-4c36-b452-d79527d7f655 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:10.764176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 589/1174] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:10.769442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:10.769833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:10.770010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:10.786939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:10.788151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:10.850560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 586/1175] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:26:10.875233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84978428-210c-4c36-b452-d79527d7f655 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:10.930779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:10.972792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84978428-210c-4c36-b452-d79527d7f655 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:11.003758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c86a5b-64e1-4d00-b486-cca4ad015050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:11.004591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c86a5b-64e1-4d00-b486-cca4ad015050 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:11.004868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c86a5b-64e1-4d00-b486-cca4ad015050 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:11.005136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c86a5b-64e1-4d00-b486-cca4ad015050 None admin] PATH_INFO: `/v3/domains/fe22baa411a9427fb27377880da93aa0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:11.006273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c86a5b-64e1-4d00-b486-cca4ad015050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=fe22baa411a9427fb27377880da93aa0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:11.007702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c86a5b-64e1-4d00-b486-cca4ad015050 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:11.009259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 590/1176] 10.222.0.41 () {56 vars in 1183 bytes} [Mon Jun 6 20:26:10 2022] GET /identity/v3/domains/fe22baa411a9427fb27377880da93aa0 => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:26:11.015522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:11.056006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:11.109657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e33ed-6b95-403a-89a6-c9e1bf183995 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:11.141779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5eaec-051b-4895-b3ab-fae695e7cb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:11.142417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5eaec-051b-4895-b3ab-fae695e7cb6e None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:11.142696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5eaec-051b-4895-b3ab-fae695e7cb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:11.142968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5eaec-051b-4895-b3ab-fae695e7cb6e None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:11.143687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5eaec-051b-4895-b3ab-fae695e7cb6e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:11.145097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5eaec-051b-4895-b3ab-fae695e7cb6e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:11.222494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 587/1177] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:26:11 2022] POST /identity/v3/users => generated 311 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:26:13.483737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698311-f87f-4896-bb52-8e701255f1d0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:13.484247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698311-f87f-4896-bb52-8e701255f1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:13.484247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c698311-f87f-4896-bb52-8e701255f1d0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:13.485554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 591/1178] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:13.490831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:13.491215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:13.491398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:13.507596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:13.508824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:13.572119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 588/1179] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:26:13.596128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698311-f87f-4896-bb52-8e701255f1d0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:13.693744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c698311-f87f-4896-bb52-8e701255f1d0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:13.715944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:13.725987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:13.726758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:13.729021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:13.729021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:13.738736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:26:13.746370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:26:13.747368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:26:13.748747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 592/1180] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:26:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:26:13.753586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:13.779912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:13.847748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c999c-73f6-4d10-8293-bcaf71419405 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:13.880527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:13.881179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:13.881466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:13.881743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:13.882478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:13.882784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:13.884180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:13.892488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 589/1181] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:26:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:26:13.898356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:13.991909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053683e-86a6-4f77-9dff-4169b5875c69 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:14.023669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:14.024526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:14.024810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:14.025083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] PATH_INFO: `/v3/users/9eac01ee09ca401eb1960a64f8872623` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:14.038219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] RBAC: Authorizing `identity:get_user(user_id=9eac01ee09ca401eb1960a64f8872623)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:14.039642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:14.041244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 593/1182] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:26:13 2022] GET /identity/v3/users/9eac01ee09ca401eb1960a64f8872623 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:14.046972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:14.133568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:14.143314 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3df7ff-2007-4477-a549-b792ab00d66d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:14.176437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:14.177065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:14.177354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:14.177630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:14.243169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:14.244594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:14.328611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:14.338266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 590/1183] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:26:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:26:14.344330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:14.385350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:14.438754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a6f94-165f-4848-aeb4-e4ddb1b8b57a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:14.470951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdbfdcb-8141-4dc6-95a4-93c398a7e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:14.471809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbfdcb-8141-4dc6-95a4-93c398a7e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:14.472088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbfdcb-8141-4dc6-95a4-93c398a7e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:14.472359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdbfdcb-8141-4dc6-95a4-93c398a7e3d1 None admin] PATH_INFO: `/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:14.473499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbfdcb-8141-4dc6-95a4-93c398a7e3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=ce8f3c62a7944a37ba84bb180b8332a8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:14.474901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdbfdcb-8141-4dc6-95a4-93c398a7e3d1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:14.476554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 594/1184] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:26:14 2022] GET /identity/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:14.482514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:14.576337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edaf3b-c0e2-4ba5-be5d-a04755e90c89 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:14.608302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:14.608929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:14.609215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:14.609491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:14.610302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=ce8f3c62a7944a37ba84bb180b8332a8, user.id=9eac01ee09ca401eb1960a64f8872623) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:14.610611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:14.612057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d63e88-6c52-4546-a98e-edeb27a5defa None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:14.617216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 591/1185] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:26:14 2022] GET /identity/v3/role_assignments?user.id=9eac01ee09ca401eb1960a64f8872623&scope.project.id=ce8f3c62a7944a37ba84bb180b8332a8&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:16.777388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c4108-42a2-48d2-ae7d-64e95e96be04 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:16.777924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c4108-42a2-48d2-ae7d-64e95e96be04 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:16.777924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c4108-42a2-48d2-ae7d-64e95e96be04 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:16.779259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 595/1186] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:16.784888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:16.785276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:16.785445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:16.801561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:16.802751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:16.849845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:16.866747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 592/1187] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:26:16.889653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c4108-42a2-48d2-ae7d-64e95e96be04 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:16.984841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c4108-42a2-48d2-ae7d-64e95e96be04 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:16.994839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:17.016968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:17.017723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:17.018002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:17.018271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:17.028320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:26:17.035982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:26:17.037048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:26:17.038239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 596/1188] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:26:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:26:17.043408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:17.136469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc9784-6afe-4122-9887-4df8fc0354cf None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:17.168671 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:17.169312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:17.169598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:17.169874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:17.170603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:17.170907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:17.172294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:17.179949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 593/1189] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:26:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:17.186184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:17.281485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3883aeee-8196-44ff-bee4-ceebeca31fd0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:17.313677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:17.314528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:17.314804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:17.315070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] PATH_INFO: `/v3/users/9eac01ee09ca401eb1960a64f8872623` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:17.316244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] RBAC: Authorizing `identity:get_user(user_id=9eac01ee09ca401eb1960a64f8872623)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:17.317661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:17.319238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 597/1190] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:26:17 2022] GET /identity/v3/users/9eac01ee09ca401eb1960a64f8872623 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:17.325258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:17.370269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:17.421049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c63ca-59ce-46c5-aca8-056582f12c53 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:17.452089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3199a7d-ad9d-4bdc-8112-dc71b7c7b783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:17.452931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199a7d-ad9d-4bdc-8112-dc71b7c7b783 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:17.453210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199a7d-ad9d-4bdc-8112-dc71b7c7b783 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:17.453481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199a7d-ad9d-4bdc-8112-dc71b7c7b783 None admin] PATH_INFO: `/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:17.454628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3199a7d-ad9d-4bdc-8112-dc71b7c7b783 None admin] RBAC: Authorizing `identity:get_project(project_id=ce8f3c62a7944a37ba84bb180b8332a8)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:17.456071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3199a7d-ad9d-4bdc-8112-dc71b7c7b783 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:17.457707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 594/1191] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:26:17 2022] GET /identity/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:17.463439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:17.501912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:17.558789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f89cb4-a2f8-40a8-b860-7080d4e4ae6c None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:17.590496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4684920-480f-422b-823f-749b1d1bf665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:17.591344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4684920-480f-422b-823f-749b1d1bf665 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:17.591618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4684920-480f-422b-823f-749b1d1bf665 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:17.591913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4684920-480f-422b-823f-749b1d1bf665 None admin] PATH_INFO: `/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8/users/9eac01ee09ca401eb1960a64f8872623/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:17.593810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4684920-480f-422b-823f-749b1d1bf665 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce8f3c62a7944a37ba84bb180b8332a8, user_id=9eac01ee09ca401eb1960a64f8872623, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:17.595217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4684920-480f-422b-823f-749b1d1bf665 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:17.660812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 598/1192] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:26:17 2022] PUT /identity/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8/users/9eac01ee09ca401eb1960a64f8872623/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:26:19.860222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5222d8-b2e5-4849-a3a5-03648587596f None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:19.860798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5222d8-b2e5-4849-a3a5-03648587596f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:19.860798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5222d8-b2e5-4849-a3a5-03648587596f None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:19.862119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 595/1193] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:26:19.867234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:19.867608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:19.867834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:19.885395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:26:19.886612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:26:19.951143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 599/1194] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:26:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:26:19.974452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5222d8-b2e5-4849-a3a5-03648587596f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:20.071100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5222d8-b2e5-4849-a3a5-03648587596f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:20.083669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:20.101964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:20.102726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:20.105851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:20.105851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:20.113785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:26:20.121767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:26:20.122921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:26:20.123837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 596/1195] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:26:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:26:20.128812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:20.154512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:20.224656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140c430-f99e-41fe-a3b7-253c20e73a42 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:20.257449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:20.258103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:20.258385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:20.258656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:20.259388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:20.259720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:20.261093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:20.268841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 600/1196] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:26:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:20.274489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:20.370478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070f75a-52cd-438b-8054-2953f9f732bc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:20.402135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:20.403504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:20.403818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:20.404096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] PATH_INFO: `/v3/users/9eac01ee09ca401eb1960a64f8872623` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:20.405292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] RBAC: Authorizing `identity:get_user(user_id=9eac01ee09ca401eb1960a64f8872623)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:20.406998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:20.408624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 597/1197] 10.222.0.41 () {56 vars in 1177 bytes} [Mon Jun 6 20:26:20 2022] GET /identity/v3/users/9eac01ee09ca401eb1960a64f8872623 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:20.414349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:20.499357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:20.508322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cd1c6-8cbe-476b-ab8d-fb1212455204 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:20.540484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:20.541101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:20.541377 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:20.541648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:20.604478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:20.606473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:20.689035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:20.698226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 601/1198] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:26:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:26:20.704535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:20.745941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:20.799829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a7877f-e10e-46d7-9ffe-e036eaa619d3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:20.831976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd926d3b-e91f-41c8-9ae4-8dbf71dca027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:20.832814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd926d3b-e91f-41c8-9ae4-8dbf71dca027 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:20.833100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd926d3b-e91f-41c8-9ae4-8dbf71dca027 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:20.833380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd926d3b-e91f-41c8-9ae4-8dbf71dca027 None admin] PATH_INFO: `/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:20.834543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd926d3b-e91f-41c8-9ae4-8dbf71dca027 None admin] RBAC: Authorizing `identity:get_project(project_id=ce8f3c62a7944a37ba84bb180b8332a8)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:20.835993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd926d3b-e91f-41c8-9ae4-8dbf71dca027 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:20.837642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 598/1199] 10.222.0.41 () {56 vars in 1186 bytes} [Mon Jun 6 20:26:20 2022] GET /identity/v3/projects/ce8f3c62a7944a37ba84bb180b8332a8 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:20.844146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:26:20.937729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0190bf-e11a-4052-bbd9-73c774ae97bc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:26:20.970437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:26:20.971068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:26:20.971351 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:26:20.971624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:26:20.972462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=ce8f3c62a7944a37ba84bb180b8332a8, user.id=9eac01ee09ca401eb1960a64f8872623) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:26:20.972768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:26:20.974163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359de6db-28c2-4b5a-a201-181f996fb904 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:26:20.979737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 602/1200] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:26:20 2022] GET /identity/v3/role_assignments?user.id=9eac01ee09ca401eb1960a64f8872623&scope.project.id=ce8f3c62a7944a37ba84bb180b8332a8&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:26:30.827970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=45 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:26:30.977166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=73 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:35.715251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d21ced-6f1d-4f0a-95e0-8a88efb88fc1 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:35.715940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d21ced-6f1d-4f0a-95e0-8a88efb88fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:35.715940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d21ced-6f1d-4f0a-95e0-8a88efb88fc1 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:35.717153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 599/1201] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:35.722513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:35.722953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:35.723063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:35.740082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:35.741249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:27:35.806645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 603/1202] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:27:35.831773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d21ced-6f1d-4f0a-95e0-8a88efb88fc1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:35.931551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d21ced-6f1d-4f0a-95e0-8a88efb88fc1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:35.964425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:35.965826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:35.966058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:35.966332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:35.975945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:35.977375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:35.994152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:35.995291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:35.996489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:35.996489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:35.996489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:35.996489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:35.996489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 600/1203] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:36.001355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:36.096961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e27d78-95ab-449f-b30d-c5f642d6bb6a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:36.123488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:36.134809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:36.135462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:36.135767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:36.136042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:36.137272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:36.137571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:36.138936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a035286-6607-4c9a-bf5f-8ccf7dbe66ad None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:36.150152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 604/1204] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:36.156762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:36.207458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:36.253034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c98ce-c6e1-4382-aaca-9299a1266b4b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:36.286140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad411bd-2979-4935-a7c0-d63ee12f6bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:36.286798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad411bd-2979-4935-a7c0-d63ee12f6bd4 None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:36.287082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad411bd-2979-4935-a7c0-d63ee12f6bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:36.287357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad411bd-2979-4935-a7c0-d63ee12f6bd4 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:36.288088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad411bd-2979-4935-a7c0-d63ee12f6bd4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:36.289521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad411bd-2979-4935-a7c0-d63ee12f6bd4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:36.420673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 601/1205] 10.222.0.41 () {60 vars in 1132 bytes} [Mon Jun 6 20:27:36 2022] POST /identity/v3/users => generated 251 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:27:38.665214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f9450-8b49-4f47-8d72-e449b91f980e None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:38.665733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f9450-8b49-4f47-8d72-e449b91f980e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:38.665733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f9450-8b49-4f47-8d72-e449b91f980e None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:38.667042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 605/1206] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:38.673118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:38.680390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:38.680390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:38.690126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:38.691300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:27:38.755154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 602/1207] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:27:38.778904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f9450-8b49-4f47-8d72-e449b91f980e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:38.881367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f9450-8b49-4f47-8d72-e449b91f980e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:38.913787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:38.914561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:38.914839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:38.915111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:38.925773 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:27:38.933393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:38.937767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:38.938784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 606/1208] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:27:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:38.940801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:38.980708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:39.037154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7886935-3fa3-4865-b595-ee43e6369d25 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:39.068820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:39.069471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:39.069752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:39.070027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:39.070750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:39.071052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:39.072452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:39.080653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 603/1209] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:27:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:39.086677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:39.134786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:39.181918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f8e62-051a-41d2-a918-fd260e9afda8 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:39.213600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:39.214398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:39.214653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:39.215476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:39.224993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:39.226437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:39.244733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:39.245873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:39.247133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:39.247133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:39.247133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:39.247133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:39.247133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 607/1210] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:39.251709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:39.345686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:39.348497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd260-4626-4580-b11d-86a7ab068aaf None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:39.380498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:39.381150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:39.381437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:39.381716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:39.382426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:39.382734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:39.384130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:39.395993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 604/1211] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:39.401524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:39.499102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a23f5-5773-4884-887c-41592f206e0d None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:39.500631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:39.531939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221462-a869-4226-b965-438c66af542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:39.532722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221462-a869-4226-b965-438c66af542f None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:39.533007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221462-a869-4226-b965-438c66af542f None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:39.533286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221462-a869-4226-b965-438c66af542f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:39.540515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab221462-a869-4226-b965-438c66af542f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:39.541959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab221462-a869-4226-b965-438c66af542f None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ab221462-a869-4226-b965-438c66af542f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:27:39.556673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:39.561002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:39.562149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 608/1212] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:39.564083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:39.660756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b23df58-c51c-444d-81c5-385775496098 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:39.693151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:39.693810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:39.694094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:39.694371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:39.695186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:39.695497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:39.696952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:39.713727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 605/1213] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 20:27:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:39.719479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221462-a869-4226-b965-438c66af542f None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:39.818187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221462-a869-4226-b965-438c66af542f None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:39.850026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:39.850649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:39.850929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:39.851201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:39.880107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:39.916184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:39.917653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:40.013142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 609/1214] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:27:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:27:40.018816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:40.037793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:40.113350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92c7d7-eac6-407a-a7ae-98ed165d9789 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:40.146807 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:40.147576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:40.147884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:40.148156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:40.159526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:40.162263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:40.182811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:40.184477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:40.185484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:40.185484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:40.185484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:40.185484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:40.185484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 606/1215] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:40.190408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:40.248825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:40.287296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d8595-eb62-4278-bbe1-315af9278f22 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:40.322770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:40.323604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:40.324075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:40.325386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:40.326122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:40.326431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:40.327852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:40.338923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 610/1216] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:40.344648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:40.412244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:40.441131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb988758-db6f-40fc-b1f2-597e5c7e13eb None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:40.474382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:40.475146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:40.475427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:40.475721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:40.482598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:40.489445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:27:40.498080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:40.499134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:40.500113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 607/1217] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:27:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:40.505038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:40.599244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ac61b-1a62-415d-8b47-349455e124ac None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:40.620371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:40.631966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:40.632631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:40.632909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:40.633178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:40.633949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:40.634254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:40.635686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856a9f7-8e3a-4bef-8410-b4b2474ee986 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:40.647433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 611/1218] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:27:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:40.653666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:40.751508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68876a-0756-4e2b-9ac1-e14f7c8249a4 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:40.778332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:40.784270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:40.784909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:40.785193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:40.785470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:40.786287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=4dd69635d9a5418e818de21ca309f9a1) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:40.786597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:40.788040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3bb072-7bf2-4aed-955c-9f36eb5361ed None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:40.793791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 608/1219] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:27:40 2022] GET /identity/v3/role_assignments?user.id=4dd69635d9a5418e818de21ca309f9a1&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:42.981593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a65a7-ee8e-4de2-962d-d5b61168e6be None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:42.982150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a65a7-ee8e-4de2-962d-d5b61168e6be None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:42.982150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a65a7-ee8e-4de2-962d-d5b61168e6be None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:42.983515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 612/1220] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:42.988551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:42.988946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:42.989112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:43.006068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:43.013569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:27:43.072833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 609/1221] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:27:43.095518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a65a7-ee8e-4de2-962d-d5b61168e6be None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:43.193708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a65a7-ee8e-4de2-962d-d5b61168e6be None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:43.226013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:43.226790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:43.227078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:43.227348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:43.238082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:27:43.245550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:43.246843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:43.248067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 613/1222] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:27:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:43.252645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:43.350114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3923f7ae-509e-4430-b09d-585a529c626a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:43.372652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:43.382719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:43.383373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:43.383654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:43.383960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:43.384695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] RBAC: Adding query filter params (name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:43.384999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:43.386356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:43.393905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 610/1223] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:27:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:43.399657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:43.445831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:43.500023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c133a6-bf13-4b0b-be63-cc5860f59e66 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:43.532858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:43.533628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:43.533909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:43.534181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:43.544210 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:43.545646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:43.562567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:43.564494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:43.565386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:43.565386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:43.565386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:43.565386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:43.565386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 614/1224] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:43.569508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:43.667682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ec915-4264-41fd-bfa9-e30dea31d749 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:43.700108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:43.700751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:43.701040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:43.701318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:43.702025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:43.702331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:43.703703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:43.715475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 611/1225] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:43.721152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:43.812594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:43.818958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9436c2e0-1ce0-4d40-b351-2c89c5ccc9d3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:43.851039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:43.851866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:43.852148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:43.852422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:43.860257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:43.861763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:27:43.876560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:43.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:43.878976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 615/1226] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:43.883419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:43.928268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:43.978229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37ec2b-cf36-49fa-95cb-050022bc6448 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:44.011530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:44.012808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:44.013094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:44.013375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:44.014194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:44.014507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:44.015972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:44.032871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 612/1227] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 20:27:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:44.038485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:44.135787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ee7d2-2e34-4131-80e3-427e3eed7d23 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:44.167906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:44.168683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:44.168964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:44.169240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:44.179265 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:44.180754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:44.197746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:44.198764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:44.200302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:44.200302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:44.200302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:44.200302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:44.203827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 616/1228] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:44.208303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:44.301597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c38ee3-c9e5-4fae-a88b-49b1fa0e8057 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:44.303895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:44.334452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:44.335107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:44.335392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:44.335692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:44.336406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:44.336710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:44.338074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:44.348908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 613/1229] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:44.354525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:44.364234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:44.452332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f3ad8-6338-4641-b8af-b937ce76b4c9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:44.484852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:44.485660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:44.485942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:44.486212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:44.492777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:44.494250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:27:44.508656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:44.509718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:44.511032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 617/1230] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:27:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:44.515571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:44.613649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d4d4b-7d6a-4288-8157-8ee2e6e4c74b None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:44.645666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:44.646310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:44.646590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:44.646861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:44.647633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:44.648003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:44.649454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93377358-bec8-4fcc-b736-f4e923819ed4 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:44.661907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 614/1231] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:27:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:44.667823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:44.731803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:44.763485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca560373-f445-40c1-9c99-76d527c1e795 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:44.797355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cce7ce-cb14-4843-a875-ff1c96b18b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:44.798224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce7ce-cb14-4843-a875-ff1c96b18b55 None admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:44.798499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce7ce-cb14-4843-a875-ff1c96b18b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:44.798767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cce7ce-cb14-4843-a875-ff1c96b18b55 None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/4dd69635d9a5418e818de21ca309f9a1/roles/484f70fabfa541f38ba2ad9d65277743` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:44.812740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cce7ce-cb14-4843-a875-ff1c96b18b55 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=4dd69635d9a5418e818de21ca309f9a1, role_id=484f70fabfa541f38ba2ad9d65277743)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:44.814217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cce7ce-cb14-4843-a875-ff1c96b18b55 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:44.883038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 618/1232] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:27:44 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/4dd69635d9a5418e818de21ca309f9a1/roles/484f70fabfa541f38ba2ad9d65277743 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:27:47.077903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cda65-dcae-4d43-8a2e-fdbee384450e None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.078452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cda65-dcae-4d43-8a2e-fdbee384450e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.078452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cda65-dcae-4d43-8a2e-fdbee384450e None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.079827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 615/1233] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:47.084898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.085304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.085455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.101699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:47.102884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:27:47.166964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 619/1234] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:27:47.191294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cda65-dcae-4d43-8a2e-fdbee384450e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:47.216673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:47.290339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cda65-dcae-4d43-8a2e-fdbee384450e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:47.323816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:47.324599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.324878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.325150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.340203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:27:47.342631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:47.343738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:47.344711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 616/1235] 10.222.0.41 () {56 vars in 1102 bytes} [Mon Jun 6 20:27:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:47.349824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:47.394654 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:47.447016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62b3b4-acbe-415e-b023-48dae7b26914 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:47.478245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:47.478896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.479181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.479477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.480745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:47.481056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:47.482474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:47.491275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 620/1236] 10.222.0.41 () {56 vars in 1103 bytes} [Mon Jun 6 20:27:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:47.497745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:47.580442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:47.596027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b296-291e-474d-8ba6-9b17c7dd93cd None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:47.627849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:47.628626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.628908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.629181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.639612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:47.641083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:47.658295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:47.659352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:47.660661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:47.660661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:47.660661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:47.660661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:47.660661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 617/1237] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:47.665343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:47.761269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d29e64-069c-49e2-b7bb-96479ea2a8c2 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:47.770277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:47.795040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:47.795771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.796133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.796361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.797062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:47.797360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:47.798702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:47.809508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 621/1238] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:47.815164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:47.912383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2550-5d22-4403-a11d-f61c6a72ae1e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:47.944488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:47.945269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:47.945554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:47.945827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:47.948541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:47.953966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:47.955403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:27:47.969985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:47.971184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:47.972569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 618/1239] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:47.976874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:48.075023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fdfab-f31d-45d2-aa55-c2654c495e99 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:48.108761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:48.109408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:48.109689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:48.109960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:48.110837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:48.111141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:48.112588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:48.129323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 622/1240] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 20:27:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:48.135008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:48.233250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732fcb22-b941-45e4-b7ba-61a813faa36e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:48.266446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:48.267085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:48.267369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:48.267646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:48.328034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:48.334448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:48.335930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:48.431015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 619/1241] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:27:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:27:48.437353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:48.464064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:48.535062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622a789-0ddc-4e24-9710-a6cfd3165edb None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:48.567629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:48.568448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:48.568713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:48.568982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:48.579224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:48.580697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:48.597358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:48.598389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:48.599745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:48.599745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:48.599745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:48.599745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:48.599745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 623/1242] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:48.604215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:48.694550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:48.700847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c425b5-4aa9-4c15-806e-9de2d169410e None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:48.733069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:48.733722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:48.734471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:48.734471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:48.735053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:48.735356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:48.736743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:48.748030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 620/1243] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:48.753630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:48.828317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:48.848288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadec633-0ac3-43b0-b55e-2af374bd1388 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:48.881318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:48.882100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:48.882382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:48.882652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:48.889236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:48.890668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:27:48.903916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:48.904911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:48.905788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 624/1244] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:27:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:48.911243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:49.008863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28b5c7-32aa-495d-be0d-d8d4cf0683e7 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:49.040394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:49.041031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:49.041310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:49.041578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:49.042342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:49.042644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:49.044609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbe662d-f1b7-455f-a110-8a6ae9aabad6 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:49.056949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 621/1245] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:27:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:49.063603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:49.159578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd3eee-30b3-4ecb-8e30-c4e651613ca0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:49.191703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:49.192344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:49.192627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:49.192910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:49.193719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] RBAC: Adding query filter params (role.id=484f70fabfa541f38ba2ad9d65277743, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=4dd69635d9a5418e818de21ca309f9a1) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:49.194026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:49.195429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bee105-3ec3-4beb-a21d-b46ad19854cb None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:49.201278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 625/1246] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:27:49 2022] GET /identity/v3/role_assignments?user.id=4dd69635d9a5418e818de21ca309f9a1&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=484f70fabfa541f38ba2ad9d65277743 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:51.475553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3672-d2a4-406a-bb8b-d6ec24d93234 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:51.476110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3672-d2a4-406a-bb8b-d6ec24d93234 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:51.476220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe3672-d2a4-406a-bb8b-d6ec24d93234 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:51.477522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 622/1247] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:51.483633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:51.484064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:51.484153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:51.489051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:51.501541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:51.502713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:27:51.567429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 626/1248] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:27:51.591549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe3672-d2a4-406a-bb8b-d6ec24d93234 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:51.609918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:51.689560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe3672-d2a4-406a-bb8b-d6ec24d93234 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:51.721983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:51.722755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:51.723032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:51.723304 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:51.732927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:27:51.740478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:51.741585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:51.742487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 623/1249] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:27:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:51.747764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:51.843802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e993b4-be2a-4435-978f-0eecf6dea489 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:51.871618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:51.876470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:51.877118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:51.877397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:51.877669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:51.878402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:51.878705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:51.880127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:51.888463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 627/1250] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:27:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:51.894134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:51.969254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:51.990832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4c488f-488c-4195-aaf2-7995a44342b1 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:52.022569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:52.023335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:52.023643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:52.023946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:52.034811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:52.037989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:52.052885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:52.053903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:52.055049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:52.055049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:52.055049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:52.055049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:52.055049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 624/1251] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:52.060329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:52.157376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bbd9f-0df4-4b8b-9896-e7ef3aa0c00a None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:52.189245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:52.189882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:52.190160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:52.190430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:52.191128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:52.191425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:52.192897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:52.205421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 628/1252] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:27:52.211713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:52.310551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e17f5-1426-4c68-a7d3-2c63a7e1717c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:52.335248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:52.343262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:52.344146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:52.344429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:52.344698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:52.351919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:52.353350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:27:52.367098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:52.368238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:52.369483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 625/1253] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:52.374627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:52.428767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:52.471954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60dfd7b-7551-4837-92ac-91f777d5cc18 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:52.505529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:52.506214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:52.506496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:52.507048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:52.507895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:52.508207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:52.509608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:52.526007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 629/1254] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 20:27:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:52.532062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:52.629028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d7eba-2ee7-4c1c-a5bf-cbb4d8a577fc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:52.661154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:52.661799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:52.662086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:52.662381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:52.715341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:52.726822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:52.728304 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:52.824680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 626/1255] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:27:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:27:52.830371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:52.927334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a505-0a62-4e7d-bcff-ca4c1567c508 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:52.940641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:52.961120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:52.961896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:52.962182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:52.962454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:52.972259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:52.973680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:52.990349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:52.991397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:52.992495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:52.992495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:52.992495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:52.992495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:52.992495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 630/1256] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:52.997948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:53.082735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:53.094217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e93b8e-3ff6-4d5d-aa0f-a88ce254b8ef None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:53.126712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:53.127377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:53.127688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:53.127969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:53.128701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:53.129003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:53.130357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:53.142451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 627/1257] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:53.148114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:53.243910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db246a8-5dee-491e-9154-503070d27a72 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:53.278800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:53.279577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:53.279886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:53.280159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:53.286815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:53.288307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:27:53.301526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:53.302532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:27:53.303628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 631/1258] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:27:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:53.308666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:53.405422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea571f-9252-4b9c-abfa-f7c068a8fbdc None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:53.437941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:53.438601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:53.438887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:53.439165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:53.440058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:53.440388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:53.441840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65602de-efe9-484c-81ca-44ebdd5d7979 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:53.454342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 628/1259] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:27:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:27:53.460447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:53.486353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:53.559497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf003e9-1b58-4492-a3c5-b7af2965f4c3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:53.593228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:53.593864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:53.594145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:53.594417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:53.595231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=4dd69635d9a5418e818de21ca309f9a1) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:53.595537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:53.597031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d712fe36-bc44-459c-a77c-b440a12e5ad1 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:53.602275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 632/1260] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:27:53 2022] GET /identity/v3/role_assignments?user.id=4dd69635d9a5418e818de21ca309f9a1&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:55.841909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9530f0f-e12b-4dee-9620-65f4c2a8834b None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:55.842465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9530f0f-e12b-4dee-9620-65f4c2a8834b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:55.842465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9530f0f-e12b-4dee-9620-65f4c2a8834b None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:55.844564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 629/1261] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:55.849861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:55.850252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:55.850411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:55.866615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:55.868682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:27:55.932774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 633/1262] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:27:55.956413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9530f0f-e12b-4dee-9620-65f4c2a8834b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:55.967367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:56.056077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9530f0f-e12b-4dee-9620-65f4c2a8834b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:56.090115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:56.090890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:56.091171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:56.091445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:56.101437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:27:56.108960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:56.115780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:56.116839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 630/1263] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:56.117685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:56.135111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:56.213580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee866c3-60cf-4496-9907-a98d7d62825b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:56.248118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:56.248780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:56.249065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:56.249342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:56.250086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:56.250393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:56.251787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:56.259747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 634/1264] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:27:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:56.265632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:56.331181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:56.367857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077a82d-08c6-4d71-8a60-a152da450c35 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:56.400044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:56.400815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:56.401097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:56.401370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:56.411815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:56.413255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:56.430088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:56.431147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:56.432292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:56.432292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:56.432292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:56.432292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:56.432292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 631/1265] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:56.437828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:56.516915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:56.534945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec811cb-192b-4861-aab8-5cfb5a7d6fb0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:56.568663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:56.569307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:56.569587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:56.569900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:56.570586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:56.570884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:56.572259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:56.583498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 635/1266] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:56.589204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:56.687030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6001d5-eabb-462b-ba99-a6f1a184eff5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:56.705760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:56.719194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:56.720018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:56.720297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:56.720565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:56.727581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:56.729088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:27:56.743284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:56.744417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:56.746102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 632/1267] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:56.750242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:56.847268 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae82e6-bdcb-44b3-9a00-cbe43253fca0 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:56.880892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:56.881547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:56.881827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:56.882102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:56.882915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:56.883222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:56.884669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:56.893108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:56.901216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 636/1268] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 20:27:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:56.907576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:57.003283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213049b2-db0b-438b-8cb7-7ab79da6cfb3 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:57.036566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:57.037347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:57.037629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:57.037904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:57.048131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:57.049579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:27:57.066139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:57.067161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:57.068460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:27:57.068460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:27:57.068460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:27:57.068460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:57.068460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 633/1269] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:57.073117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:57.169235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b36a80-c080-49f0-b6cb-8fb04ecb09bc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:57.203779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:57.204511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:57.204828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:57.205103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:57.205808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:57.206113 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:57.207478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:57.219101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 637/1270] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:57.225011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:57.259089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:57.325715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b330cd-1732-4dd1-b59b-c22f61bf1031 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:57.356993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:57.357771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:57.358052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:57.358324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:57.364692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:57.366108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:27:57.380551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:27:57.381655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:27:57.382604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 634/1271] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:27:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:27:57.388241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:57.445304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:57.484844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6854288-e4c3-4f56-aaf1-cc09de2b4a13 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:57.517660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:57.518242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:57.518522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:57.518796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:57.519565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:27:57.519901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:57.521311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca521e86-7026-4024-8377-14e02abb98cc None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:57.533679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 638/1272] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:27:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:27:57.539596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:27:57.627449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:27:57.635857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30cb6a-3fa9-4163-9a64-ea44baad5256 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:27:57.668455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a95bb2-865d-473b-bacd-e79cda25b9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:27:57.669325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95bb2-865d-473b-bacd-e79cda25b9ae None admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:57.669611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95bb2-865d-473b-bacd-e79cda25b9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:57.669889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a95bb2-865d-473b-bacd-e79cda25b9ae None admin] PATH_INFO: `/v3/projects/2c821c67b8a14f83b5227073f824d703/users/4dd69635d9a5418e818de21ca309f9a1/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:57.672101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a95bb2-865d-473b-bacd-e79cda25b9ae None admin] RBAC: Authorizing `identity:create_grant(project_id=2c821c67b8a14f83b5227073f824d703, user_id=4dd69635d9a5418e818de21ca309f9a1, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:27:57.673907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a95bb2-865d-473b-bacd-e79cda25b9ae None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:27:57.746450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 635/1273] 10.222.0.41 () {58 vars in 1439 bytes} [Mon Jun 6 20:27:57 2022] PUT /identity/v3/projects/2c821c67b8a14f83b5227073f824d703/users/4dd69635d9a5418e818de21ca309f9a1/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:27:59.958633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0e129-03c1-4ee4-a7b0-98b1e640b505 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:59.959217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0e129-03c1-4ee4-a7b0-98b1e640b505 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:59.959217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0e129-03c1-4ee4-a7b0-98b1e640b505 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:59.960612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 639/1274] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:27:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:27:59.966417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:27:59.966962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:27:59.966962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:27:59.985053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:27:59.986276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:28:00.057626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 636/1275] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:27:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:28:00.082390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0e129-03c1-4ee4-a7b0-98b1e640b505 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:00.187871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0e129-03c1-4ee4-a7b0-98b1e640b505 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:00.205034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:00.221793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:00.222582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:00.222863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:00.223137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:00.236858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 06 20:28:00.244658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:00.245818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:00.247341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 640/1276] 10.222.0.41 () {56 vars in 1096 bytes} [Mon Jun 6 20:28:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:00.261688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:00.341635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:00.358857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1feb5-d449-4616-b45b-bf8f59800520 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:00.391984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:00.392647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:00.392932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:00.393207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:00.393942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:00.394247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:00.395622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:00.404096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 637/1277] 10.222.0.41 () {56 vars in 1099 bytes} [Mon Jun 6 20:28:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:00.409950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:00.506179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067dfe9d-592a-41ea-bb3a-c6963946a0fc None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:00.538173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:00.538940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:00.539220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:00.539495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:00.550094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:00.551520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:28:00.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:00.568879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:00.569813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:28:00.569813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:28:00.569813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:28:00.569813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:00.569813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 641/1278] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:28:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:00.574701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:00.672404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f965a-8bd8-4091-8dd0-e79e7c7fd398 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:00.704297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:00.704988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:00.705276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:00.705552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:00.706270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:00.706580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:00.707976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:00.719960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 638/1279] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:00.725827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:00.755713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:00.822905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5258beb-c223-4354-ad85-82a1b6b7f939 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:00.856692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:00.857483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:00.857783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:00.858054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:00.865478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:00.868789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 06 20:28:00.880977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:00.882184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 06 20:28:00.883315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:00.887388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 642/1280] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:28:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:00.888579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:00.892294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:00.983485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e629d37-1a2c-4b32-91c6-f596f5aba8f5 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:01.015529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:01.016202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:01.016482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:01.016752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:01.017562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:01.017868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:01.019280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:01.036193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 639/1281] 10.222.0.41 () {56 vars in 1143 bytes} [Mon Jun 6 20:28:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:01.041845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:01.133683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:01.143724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f34ee-1138-4d57-8239-b132ee1b0ac9 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:01.175918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:01.176553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:01.176837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:01.177115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:01.243212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:01.248998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:01.330888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:01.340804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 643/1282] 10.222.0.41 () {58 vars in 1282 bytes} [Mon Jun 6 20:28:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 06 20:28:01.346505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:01.405701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:01.442983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14232e5-72b0-4515-81de-334d332ee08c None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:01.475915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:01.476705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:01.476997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:01.477277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:01.487723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:01.489178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 06 20:28:01.506574 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:01.507700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:01.508773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 06 20:28:01.508773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 06 20:28:01.508773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 06 20:28:01.508773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:28:01.508773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 640/1283] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:28:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:01.513460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:01.609300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98013a9-967f-4f6e-a802-93154a1fda50 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:01.641986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:01.642637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:01.642916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:01.643184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:01.643912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:01.644212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:01.645570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:01.656895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 644/1284] 10.222.0.41 () {56 vars in 1109 bytes} [Mon Jun 6 20:28:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:01.662364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:01.760062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feed53b-3695-4fc2-b658-d32518ec0c10 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:01.773661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:01.793279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:01.794051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:01.794338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:01.794617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:01.801045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:01.802476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 06 20:28:01.816795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:01.817960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: ERROR keystone.server.flask.application Jun 06 20:28:01.819030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 641/1285] 10.222.0.41 () {56 vars in 1111 bytes} [Mon Jun 6 20:28:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:01.823679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:01.872352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:01.921226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314e3dd-7238-4f58-a349-fa2b54e71c61 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:01.953325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:01.957168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:01.957168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:01.957168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:01.957168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:01.957168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:01.957168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc64f21-4ce6-4820-9bdc-21b68a598055 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:01.969245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 645/1286] 10.222.0.41 () {56 vars in 1148 bytes} [Mon Jun 6 20:28:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:28:01.975190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:02.070759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adfaf15-057e-46a8-8e71-edec88ae116d None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:02.103505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:02.104169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:02.104458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:02.104739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:02.106150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] RBAC: Adding query filter params (role.id=6c6c8f2c4f144911a9a411b476178fc4, scope.project.id=2c821c67b8a14f83b5227073f824d703, user.id=4dd69635d9a5418e818de21ca309f9a1) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:02.106460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:02.107918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d25b21-d56d-4e59-a80b-459959e3b774 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:02.113676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 642/1287] 10.222.0.41 () {56 vars in 1374 bytes} [Mon Jun 6 20:28:01 2022] GET /identity/v3/role_assignments?user.id=4dd69635d9a5418e818de21ca309f9a1&scope.project.id=2c821c67b8a14f83b5227073f824d703&role.id=6c6c8f2c4f144911a9a411b476178fc4 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:04.361350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93531c-d88b-4c14-91d1-0c4005c461a1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:04.361901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93531c-d88b-4c14-91d1-0c4005c461a1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:04.361901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a93531c-d88b-4c14-91d1-0c4005c461a1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:04.363247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 646/1288] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:28:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:28:04.368457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:04.368857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:04.369013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:04.386150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:28:04.387936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:28:04.416407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:04.452728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 643/1289] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:28:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:28:04.474684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93531c-d88b-4c14-91d1-0c4005c461a1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:04.572938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a93531c-d88b-4c14-91d1-0c4005c461a1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:04.583065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:04.604806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:04.606173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:04.606401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:04.606672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:04.607311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:04.608703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:04.620099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:04.621898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:04.622743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 647/1290] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:28:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:04.627834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:04.722464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c6be5-713a-4dcb-aa93-ec36a313665b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:04.755397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:04.756067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:04.756349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:04.756617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:04.757296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:04.757595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:04.758941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504bdb5c-0e6b-4b4b-9caa-d4bafca85d8a None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:04.763743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 644/1291] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 20:28:04 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:04.769336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:04.867362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf7202-71b3-45f8-84f9-69f0ddc910c3 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:04.898845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:04.899485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:04.899793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:04.900069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] PATH_INFO: `/v3/services` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:04.900756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:04.901057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:04.902946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e769ee61-ab66-4417-a099-ddc02ed22b84 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:04.907517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 648/1292] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 20:28:04 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:07.107323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565e4eb-e0c4-4931-a6ce-68d5ed34dd06 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:07.107898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565e4eb-e0c4-4931-a6ce-68d5ed34dd06 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:07.107898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1565e4eb-e0c4-4931-a6ce-68d5ed34dd06 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:07.109289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 645/1293] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:28:07.114791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e2533-6b09-4d20-b082-a1412b3ceb6d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:07.115210 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e2533-6b09-4d20-b082-a1412b3ceb6d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:07.115341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e2533-6b09-4d20-b082-a1412b3ceb6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:07.132237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-de7e2533-6b09-4d20-b082-a1412b3ceb6d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:28:07.133431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-de7e2533-6b09-4d20-b082-a1412b3ceb6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:28:07.162219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:07.197900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 649/1294] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:28:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:28:07.220928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565e4eb-e0c4-4931-a6ce-68d5ed34dd06 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:07.251632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:07.320271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1565e4eb-e0c4-4931-a6ce-68d5ed34dd06 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:07.352681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f523fa-e74c-492e-a801-f84a1560f70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:07.353330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f523fa-e74c-492e-a801-f84a1560f70f None admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:07.353617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f523fa-e74c-492e-a801-f84a1560f70f None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:07.353897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f523fa-e74c-492e-a801-f84a1560f70f None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:07.354618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f523fa-e74c-492e-a801-f84a1560f70f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:07.356023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f523fa-e74c-492e-a801-f84a1560f70f None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:07.403150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 646/1295] 10.222.0.41 () {60 vars in 1141 bytes} [Mon Jun 6 20:28:07 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:28:09.646294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698edc-ca2e-4fa7-bd2c-64ca997b397d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:09.646839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698edc-ca2e-4fa7-bd2c-64ca997b397d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:09.646839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698edc-ca2e-4fa7-bd2c-64ca997b397d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:09.648190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 650/1296] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:28:09.653862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:09.654294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:09.654421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:09.671595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:28:09.672840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:28:09.743529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 647/1297] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:28:09 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:28:09.766820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698edc-ca2e-4fa7-bd2c-64ca997b397d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:09.864622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698edc-ca2e-4fa7-bd2c-64ca997b397d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:09.897202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:09.897953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:09.898231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:09.898502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:09.899149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:09.900523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:09.912906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:09.914083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:09.915339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 651/1298] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:28:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:09.920221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:09.938150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:10.143910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3c373-18f9-462c-92c7-a393e8492093 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:10.176393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:10.176983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:10.177265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:10.177538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:10.178223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:10.178523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:10.179932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:10.185558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 648/1299] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 20:28:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:10.191579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:10.222225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:10.288364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501605d0-0b1d-4e5c-8cca-e9ba3a54ac90 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:10.322940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:10.323584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:10.323906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:10.324181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:10.325014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] RBAC: Adding query filter params (interface=public, service_id=cf4be3dfee644bf1b797c53bf43c517c, region_id=RegionOne) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:10.325829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:10.327187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2cef0-50c7-4acc-8247-2db1b4b01c0e None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:10.332097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 652/1300] 10.222.0.41 () {56 vars in 1251 bytes} [Mon Jun 6 20:28:10 2022] GET /identity/v3/endpoints?service_id=cf4be3dfee644bf1b797c53bf43c517c&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:10.337725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:10.415135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:10.432588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2314f3e3-ff1a-4213-be89-381077025616 None admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:10.465019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44922e-ec28-4586-9dbe-fecde9ea8b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:10.465692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44922e-ec28-4586-9dbe-fecde9ea8b05 None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:10.465974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44922e-ec28-4586-9dbe-fecde9ea8b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:10.466248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44922e-ec28-4586-9dbe-fecde9ea8b05 None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:10.466917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44922e-ec28-4586-9dbe-fecde9ea8b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:10.468303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44922e-ec28-4586-9dbe-fecde9ea8b05 None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:10.474910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 649/1301] 10.222.0.41 () {56 vars in 1087 bytes} [Mon Jun 6 20:28:10 2022] GET /identity/v3/services => generated 2191 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:28:12.692713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c260d1-1689-44b1-b1a9-7cf68a99791a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:12.693223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c260d1-1689-44b1-b1a9-7cf68a99791a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:12.693223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c260d1-1689-44b1-b1a9-7cf68a99791a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:12.694561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 653/1302] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:28:12.699642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:12.700148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:12.700243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:12.716244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:28:12.717398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:28:12.781456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 650/1303] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:28:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 06 20:28:12.804638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c260d1-1689-44b1-b1a9-7cf68a99791a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:12.897776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:12.904173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c260d1-1689-44b1-b1a9-7cf68a99791a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:12.936992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:12.937776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:12.938058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:12.938331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:12.939083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:12.940459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 06 20:28:12.952070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 06 20:28:12.953219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.get_or_create( Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application with Lock( Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self._enter() Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application return self.creator() Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application created_value = creator( Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 06 20:28:12.954401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: ERROR keystone.server.flask.application Jun 06 20:28:12.956048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 654/1304] 10.222.0.41 () {56 vars in 1117 bytes} [Mon Jun 6 20:28:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 06 20:28:12.959233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:13.033694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:13.055543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c096bba-4803-4e5e-ad69-6766ad954e97 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:13.086772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:13.087418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:13.087724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:13.088043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] PATH_INFO: `/v3/services` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:13.088846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:28:13.089146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:13.091018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d215a09-e87b-432c-a1f7-baab2e88885b None admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:13.096082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 651/1305] 10.222.0.41 () {56 vars in 1116 bytes} [Mon Jun 6 20:28:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:28:13.101828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:28:13.200702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b22900-af33-4306-a58b-2ce8d93e9a56 None admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:28:13.232859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7bfba-3d3a-4c34-9167-44b66442aa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:28:13.233504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7bfba-3d3a-4c34-9167-44b66442aa48 None admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:28:13.233788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7bfba-3d3a-4c34-9167-44b66442aa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:28:13.234064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7bfba-3d3a-4c34-9167-44b66442aa48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:28:13.234804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7bfba-3d3a-4c34-9167-44b66442aa48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:28:13.236273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7bfba-3d3a-4c34-9167-44b66442aa48 None admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:28:13.299952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 655/1306] 10.222.0.41 () {60 vars in 1144 bytes} [Mon Jun 6 20:28:13 2022] POST /identity/v3/endpoints => generated 331 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:28:23.123726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=30 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:28:23.295964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:43.841695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7926e52-cc9a-479f-92c1-073b43efd381 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:43.843024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7926e52-cc9a-479f-92c1-073b43efd381 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:43.843024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7926e52-cc9a-479f-92c1-073b43efd381 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:43.848954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ab18b-8f45-4331-aa55-1c20fad354d4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:43.849400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ab18b-8f45-4331-aa55-1c20fad354d4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:43.849494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515ab18b-8f45-4331-aa55-1c20fad354d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:43.852887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 652/1307] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:33:43 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:33:43.941141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-515ab18b-8f45-4331-aa55-1c20fad354d4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:33:43.942403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-515ab18b-8f45-4331-aa55-1c20fad354d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:44.130147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 656/1308] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:33:44.172265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e71e18-6898-4bc2-913a-a3569ec12c25 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:44.172561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e71e18-6898-4bc2-913a-a3569ec12c25 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:44.172846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e71e18-6898-4bc2-913a-a3569ec12c25 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:44.174532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 653/1309] 10.222.0.41 () {54 vars in 964 bytes} [Mon Jun 6 20:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:33:44.180958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7829b-1b39-4338-adc5-497af1b51bad None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:44.181256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7829b-1b39-4338-adc5-497af1b51bad None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:44.181545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c7829b-1b39-4338-adc5-497af1b51bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:44.299861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f1c7829b-1b39-4338-adc5-497af1b51bad None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:33:44.301346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f1c7829b-1b39-4338-adc5-497af1b51bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:44.345333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:44.409283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 657/1310] 10.222.0.41 () {58 vars in 1061 bytes} [Mon Jun 6 20:33:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:33:44.418943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e71e18-6898-4bc2-913a-a3569ec12c25 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:33:44.420962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f3e71e18-6898-4bc2-913a-a3569ec12c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:44.535277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e71e18-6898-4bc2-913a-a3569ec12c25 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:33:44.613156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:33:44.613909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:44.614262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:44.614609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:44.616371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:44.675172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:44.731892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:33:44.733511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee138d-3e70-41d3-a8d3-35528c742f7a service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:33:44.753560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 654/1311] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:33:47.312688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedf3e0-4636-4015-b968-3269219fe886 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:47.313226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedf3e0-4636-4015-b968-3269219fe886 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:47.313226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daedf3e0-4636-4015-b968-3269219fe886 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:47.314726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 658/1312] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:33:47.320144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185e6a5-34cf-41e9-82c2-b0fc93cb3c8b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:47.320576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185e6a5-34cf-41e9-82c2-b0fc93cb3c8b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:47.320803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185e6a5-34cf-41e9-82c2-b0fc93cb3c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:47.392569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6185e6a5-34cf-41e9-82c2-b0fc93cb3c8b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:33:47.393789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6185e6a5-34cf-41e9-82c2-b0fc93cb3c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:47.494193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 655/1313] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:33:47.533101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedf3e0-4636-4015-b968-3269219fe886 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:33:47.534767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-daedf3e0-4636-4015-b968-3269219fe886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:47.781092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daedf3e0-4636-4015-b968-3269219fe886 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:33:47.857378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:47.860107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:33:47.860754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:47.861029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:47.861302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:47.862940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:47.973775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:33:47.975212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2da3b-1895-4703-b355-30e5d43c6a1b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:33:47.995596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 659/1314] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:33:50.797682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc128f-b5d1-4d7f-9acc-9fda69556416 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:50.798211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc128f-b5d1-4d7f-9acc-9fda69556416 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:50.798329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fc128f-b5d1-4d7f-9acc-9fda69556416 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:50.799642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 656/1315] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:33:50.804773 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219206d4-7b98-447a-a384-216811bde5c3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:50.805200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219206d4-7b98-447a-a384-216811bde5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:50.805319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219206d4-7b98-447a-a384-216811bde5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:50.867059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-219206d4-7b98-447a-a384-216811bde5c3 None None] MFA Rules not processed for user `22fb0cc1f4074e9e9e9005557751064f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:33:50.868257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-219206d4-7b98-447a-a384-216811bde5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:50.918361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:50.961545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 660/1316] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:33:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:33:51.001528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc128f-b5d1-4d7f-9acc-9fda69556416 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:33:51.003053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c8fc128f-b5d1-4d7f-9acc-9fda69556416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:51.012674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:51.104103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fc128f-b5d1-4d7f-9acc-9fda69556416 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:33:51.176162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:33:51.176798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:51.177090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:51.177368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:51.178986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:51.238814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:51.285621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:33:51.287095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed019416-205b-4b1f-9acc-a23a6d7cc2c5 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:33:51.305054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 657/1317] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:33:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:33:54.339125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573508-df8b-409b-affd-55f57a1b3c22 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:54.339642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573508-df8b-409b-affd-55f57a1b3c22 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:54.339783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1573508-df8b-409b-affd-55f57a1b3c22 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:54.341061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 661/1318] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:33:54.346744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ab25e-464f-48cf-ae03-bfd3eb9b7b4d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:54.347124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ab25e-464f-48cf-ae03-bfd3eb9b7b4d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:54.347333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ab25e-464f-48cf-ae03-bfd3eb9b7b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:54.410058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-146ab25e-464f-48cf-ae03-bfd3eb9b7b4d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:33:54.411249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-146ab25e-464f-48cf-ae03-bfd3eb9b7b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:54.503607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 658/1319] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:33:54 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:33:54.541987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573508-df8b-409b-affd-55f57a1b3c22 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:33:54.543512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c1573508-df8b-409b-affd-55f57a1b3c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:54.643426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1573508-df8b-409b-affd-55f57a1b3c22 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:33:54.713600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:33:54.714232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:54.714511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:54.714781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:54.716426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:54.723581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:54.820971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:33:54.822313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb3371-7f7e-4aad-9e6c-ccf4e1904d10 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:33:54.840006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 662/1320] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:33:54 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:33:58.145739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e5a0-cc77-4d6a-a860-82f6f528a2ab None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:58.146264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e5a0-cc77-4d6a-a860-82f6f528a2ab None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:58.146346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e825e5a0-cc77-4d6a-a860-82f6f528a2ab None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:58.147656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 659/1321] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:33:58.153091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff7fec-ea2a-4c3d-ae46-ebaa93cf253f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:58.153491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff7fec-ea2a-4c3d-ae46-ebaa93cf253f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:58.153642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff7fec-ea2a-4c3d-ae46-ebaa93cf253f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:58.217732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7aff7fec-ea2a-4c3d-ae46-ebaa93cf253f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:33:58.218921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7aff7fec-ea2a-4c3d-ae46-ebaa93cf253f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:58.278658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:58.315714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 663/1322] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:33:58.351706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e5a0-cc77-4d6a-a860-82f6f528a2ab None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:33:58.353450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e825e5a0-cc77-4d6a-a860-82f6f528a2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:58.372248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:58.458320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e825e5a0-cc77-4d6a-a860-82f6f528a2ab None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:33:58.533492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:33:58.534159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:33:58.534447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:33:58.534725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:33:58.536562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:33:58.609897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:33:58.646990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:33:58.648519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974d1b6-0366-4109-88d3-ce5ba1464834 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:33:58.666554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 660/1323] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:00.963217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f7024-274c-47f4-af96-e8bb9ce6a34c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:00.963825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f7024-274c-47f4-af96-e8bb9ce6a34c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:00.963825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f7024-274c-47f4-af96-e8bb9ce6a34c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:00.965152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 664/1324] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:00.971003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be9d28-7323-4165-ac42-2fb5fa54520f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:00.971430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be9d28-7323-4165-ac42-2fb5fa54520f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:00.971567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96be9d28-7323-4165-ac42-2fb5fa54520f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:01.036251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-96be9d28-7323-4165-ac42-2fb5fa54520f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:01.037505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-96be9d28-7323-4165-ac42-2fb5fa54520f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:01.131560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 661/1325] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:01.182993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c76f-513d-40de-bc86-eb332334f2cd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:01.183386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c76f-513d-40de-bc86-eb332334f2cd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:01.183768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c76f-513d-40de-bc86-eb332334f2cd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:01.185182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 665/1326] 10.222.0.41 () {54 vars in 964 bytes} [Mon Jun 6 20:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:01.191228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c11a-45d1-447b-b302-af27159d4fd2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:01.191546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c11a-45d1-447b-b302-af27159d4fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:01.191861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0098c11a-45d1-447b-b302-af27159d4fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:01.222023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:01.266542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0098c11a-45d1-447b-b302-af27159d4fd2 None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:01.267903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0098c11a-45d1-447b-b302-af27159d4fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:01.350367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 662/1327] 10.222.0.41 () {58 vars in 1061 bytes} [Mon Jun 6 20:34:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:01.359623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c76f-513d-40de-bc86-eb332334f2cd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:01.361180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8bc1c76f-513d-40de-bc86-eb332334f2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:01.461753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c76f-513d-40de-bc86-eb332334f2cd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:01.534805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:01.535357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:01.535632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:01.535928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:01.537557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:01.553642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:01.644511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:01.645936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a45c48f-f277-4459-b22d-4f3036f7f5cf service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:01.663794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 666/1328] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:04.197857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8fe8e-cc5b-4268-9d82-08473ed6ee63 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:04.198408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8fe8e-cc5b-4268-9d82-08473ed6ee63 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:04.198408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8fe8e-cc5b-4268-9d82-08473ed6ee63 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:04.199729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 663/1329] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:04.205437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb31f50-8b93-488d-ade0-6733443c7592 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:04.205834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb31f50-8b93-488d-ade0-6733443c7592 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:04.205996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb31f50-8b93-488d-ade0-6733443c7592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:04.270821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9bb31f50-8b93-488d-ade0-6733443c7592 None None] MFA Rules not processed for user `22fb0cc1f4074e9e9e9005557751064f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:04.272047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9bb31f50-8b93-488d-ade0-6733443c7592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:04.334646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:04.364695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 667/1330] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:04.403061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8fe8e-cc5b-4268-9d82-08473ed6ee63 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:04.404655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a6a8fe8e-cc5b-4268-9d82-08473ed6ee63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:04.504554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8fe8e-cc5b-4268-9d82-08473ed6ee63 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:04.523910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:04.577039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:04.577681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:04.577963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:04.578237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:04.579896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:04.686167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:04.687594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c170c5-9ed2-4963-bc7c-e6d4b23512d0 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:04.705238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 664/1331] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:07.635800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbf0d-c121-476c-a5f5-f8e0181b5208 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:07.636297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbf0d-c121-476c-a5f5-f8e0181b5208 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:07.636297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bbf0d-c121-476c-a5f5-f8e0181b5208 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:07.637637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 668/1332] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:07.642845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f40da-1cce-4034-be03-fd7d763d6323 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:07.643256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f40da-1cce-4034-be03-fd7d763d6323 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:07.643396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f40da-1cce-4034-be03-fd7d763d6323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:07.704551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:07.707034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-013f40da-1cce-4034-be03-fd7d763d6323 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:07.708253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-013f40da-1cce-4034-be03-fd7d763d6323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:07.804636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 665/1333] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:07.840334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbf0d-c121-476c-a5f5-f8e0181b5208 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:07.841903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a75bbf0d-c121-476c-a5f5-f8e0181b5208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:07.952677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75bbf0d-c121-476c-a5f5-f8e0181b5208 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:08.031154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:08.031957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:08.032242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:08.032517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:08.034171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:08.064985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:08.153366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:08.155252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667aec3c-2f6a-4c47-9346-7a46f1f4e145 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:08.174461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 669/1334] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:10.459284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24304257-4e4e-463a-853f-24ceb6f07e7f None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:10.459895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24304257-4e4e-463a-853f-24ceb6f07e7f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:10.459895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24304257-4e4e-463a-853f-24ceb6f07e7f None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:10.461306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 666/1335] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:10.467229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c737fd-2c37-4e7c-ad79-642eda66f34e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:10.467720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c737fd-2c37-4e7c-ad79-642eda66f34e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:10.467832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c737fd-2c37-4e7c-ad79-642eda66f34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:10.530097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-14c737fd-2c37-4e7c-ad79-642eda66f34e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:10.531207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-14c737fd-2c37-4e7c-ad79-642eda66f34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:10.601604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:10.623069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 670/1336] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:10.664455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24304257-4e4e-463a-853f-24ceb6f07e7f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:10.665988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-24304257-4e4e-463a-853f-24ceb6f07e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:10.766228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24304257-4e4e-463a-853f-24ceb6f07e7f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:10.793875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:10.837872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:10.838503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:10.838783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:10.839054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:10.840707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:10.945223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:10.946653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb88a5a1-70de-4454-937a-14bf84a1f6ac service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:10.965067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 667/1337] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:13.896075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b72c7-e066-4b5c-82ec-db618ee7a3bb None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:13.896638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b72c7-e066-4b5c-82ec-db618ee7a3bb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:13.896638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b72c7-e066-4b5c-82ec-db618ee7a3bb None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:13.897979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 671/1338] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:13.903412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3582c-f048-46b5-9636-b6f6e7a9f647 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:13.903864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3582c-f048-46b5-9636-b6f6e7a9f647 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:13.904589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae3582c-f048-46b5-9636-b6f6e7a9f647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:13.967072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8ae3582c-f048-46b5-9636-b6f6e7a9f647 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:13.968302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8ae3582c-f048-46b5-9636-b6f6e7a9f647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:13.976324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:14.063239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 668/1339] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:34:14.098823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b72c7-e066-4b5c-82ec-db618ee7a3bb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:14.102186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f74b72c7-e066-4b5c-82ec-db618ee7a3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:14.211306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b72c7-e066-4b5c-82ec-db618ee7a3bb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:14.214782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:14.215416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:14.215722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:14.216002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:14.217766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:14.320088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:14.321536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2665d74-2c77-4f9e-97de-e2f761e5e7d6 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:14.332992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:14.342318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 672/1340] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:18.814253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244f197-dcec-4e59-8866-2d207bedbc01 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:18.814833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244f197-dcec-4e59-8866-2d207bedbc01 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:18.814833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8244f197-dcec-4e59-8866-2d207bedbc01 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:18.816372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 669/1341] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:18.821600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d096-57c3-41be-a03f-6ee77a820747 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:18.822004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d096-57c3-41be-a03f-6ee77a820747 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:18.822156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d096-57c3-41be-a03f-6ee77a820747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:18.873780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6065d096-57c3-41be-a03f-6ee77a820747 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:18.874915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6065d096-57c3-41be-a03f-6ee77a820747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:18.961578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 673/1342] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:19.000716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244f197-dcec-4e59-8866-2d207bedbc01 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:19.012117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8244f197-dcec-4e59-8866-2d207bedbc01 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:19.016085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674f421-3410-4e26-a6aa-e6bc57ea8d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:19.016701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674f421-3410-4e26-a6aa-e6bc57ea8d9e service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:19.016983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674f421-3410-4e26-a6aa-e6bc57ea8d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:19.017254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674f421-3410-4e26-a6aa-e6bc57ea8d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:19.025789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674f421-3410-4e26-a6aa-e6bc57ea8d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:19.027447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9674f421-3410-4e26-a6aa-e6bc57ea8d9e service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:19.037225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 670/1343] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:22.184447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d205d2-7524-432f-85a4-d238e345bb4d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:22.185013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d205d2-7524-432f-85a4-d238e345bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:22.185013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d205d2-7524-432f-85a4-d238e345bb4d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:22.186820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 674/1344] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:22.191949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944b4cc-88c9-4427-a67b-2be8e309bb57 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:22.192364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944b4cc-88c9-4427-a67b-2be8e309bb57 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:22.192624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944b4cc-88c9-4427-a67b-2be8e309bb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:22.219137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8944b4cc-88c9-4427-a67b-2be8e309bb57 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:22.220360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8944b4cc-88c9-4427-a67b-2be8e309bb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:22.228522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 671/1345] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:22.265874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d205d2-7524-432f-85a4-d238e345bb4d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:22.276346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d205d2-7524-432f-85a4-d238e345bb4d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:22.280635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bf24a-7cc3-448f-82f8-df7baafd81b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:22.281257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf24a-7cc3-448f-82f8-df7baafd81b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:22.281539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf24a-7cc3-448f-82f8-df7baafd81b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:22.281811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bf24a-7cc3-448f-82f8-df7baafd81b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:22.290512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf24a-7cc3-448f-82f8-df7baafd81b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:22.292020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bf24a-7cc3-448f-82f8-df7baafd81b1 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:22.301700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 675/1346] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:26.792235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959ace-dfde-451d-91fa-8fa69563ab7d None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:26.792765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959ace-dfde-451d-91fa-8fa69563ab7d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:26.792882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a959ace-dfde-451d-91fa-8fa69563ab7d None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:26.794171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 672/1347] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:26.799428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7fed9-aad8-40a4-a5ff-12490fcf3050 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:26.799945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7fed9-aad8-40a4-a5ff-12490fcf3050 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:26.800081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb7fed9-aad8-40a4-a5ff-12490fcf3050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:26.818277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2eb7fed9-aad8-40a4-a5ff-12490fcf3050 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:26.819700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2eb7fed9-aad8-40a4-a5ff-12490fcf3050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:26.828231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 676/1348] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:26.865903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959ace-dfde-451d-91fa-8fa69563ab7d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:26.876965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a959ace-dfde-451d-91fa-8fa69563ab7d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:26.880421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a4148-73f9-4d8d-bebf-6d74e26e3345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:26.881048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4148-73f9-4d8d-bebf-6d74e26e3345 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:26.881335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4148-73f9-4d8d-bebf-6d74e26e3345 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:26.881613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a4148-73f9-4d8d-bebf-6d74e26e3345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:26.890833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a4148-73f9-4d8d-bebf-6d74e26e3345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:26.892285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a4148-73f9-4d8d-bebf-6d74e26e3345 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:26.901757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 673/1349] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:32.370689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d29ec6-f139-43b0-afe8-99524533f787 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:32.371780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d29ec6-f139-43b0-afe8-99524533f787 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:32.372292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d29ec6-f139-43b0-afe8-99524533f787 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:32.374159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 677/1350] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:34:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:32.378804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795423-5c1d-4894-b7ad-da70140f4c17 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:32.379630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795423-5c1d-4894-b7ad-da70140f4c17 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:32.380095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22795423-5c1d-4894-b7ad-da70140f4c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:32.449711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-22795423-5c1d-4894-b7ad-da70140f4c17 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:32.451686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-22795423-5c1d-4894-b7ad-da70140f4c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:32.497288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:32.539834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 674/1351] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:34:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:36.791632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b5fb8-7980-46fe-a7dd-f52ed28001f2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:36.792587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b5fb8-7980-46fe-a7dd-f52ed28001f2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:36.792864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390b5fb8-7980-46fe-a7dd-f52ed28001f2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:36.795066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 678/1352] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:34:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:36.799958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab2595-62af-4881-9189-a24185223c35 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:36.800477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab2595-62af-4881-9189-a24185223c35 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:36.800591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab2595-62af-4881-9189-a24185223c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:36.812922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=56 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:36.821536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-31ab2595-62af-4881-9189-a24185223c35 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:36.822780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-31ab2595-62af-4881-9189-a24185223c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:36.832349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 675/1353] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:34:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:38.358517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7cfb-fc13-4bc2-8870-ae01c20f2263 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:38.359082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7cfb-fc13-4bc2-8870-ae01c20f2263 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:38.359182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7cfb-fc13-4bc2-8870-ae01c20f2263 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:38.360635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 679/1354] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:38.366004 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb808f-fc38-4632-9c07-ca2d19a2fb06 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:38.366422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb808f-fc38-4632-9c07-ca2d19a2fb06 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:38.366706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb808f-fc38-4632-9c07-ca2d19a2fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:38.385280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c7bb808f-fc38-4632-9c07-ca2d19a2fb06 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:38.386556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c7bb808f-fc38-4632-9c07-ca2d19a2fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:38.395422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 676/1355] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:34:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:42.243227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848fe5-4c1c-45d6-8ef3-1888574ac13b None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:42.243929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848fe5-4c1c-45d6-8ef3-1888574ac13b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:42.243929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77848fe5-4c1c-45d6-8ef3-1888574ac13b None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:42.245138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 680/1356] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:42.250886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb3df-6871-4e5e-859a-ef7a03e15693 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:42.251316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb3df-6871-4e5e-859a-ef7a03e15693 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:42.251440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865fb3df-6871-4e5e-859a-ef7a03e15693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:42.269061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-865fb3df-6871-4e5e-859a-ef7a03e15693 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:42.270253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-865fb3df-6871-4e5e-859a-ef7a03e15693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:42.278274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 677/1357] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:42.334162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e9757-57b6-471d-9ed9-5977a75e5b77 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:42.334460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e9757-57b6-471d-9ed9-5977a75e5b77 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:42.335172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e9757-57b6-471d-9ed9-5977a75e5b77 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:42.337149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 681/1358] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 20:34:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:42.343570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ddbf5-3bc7-4d88-911b-b5f72a149527 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:42.344053 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ddbf5-3bc7-4d88-911b-b5f72a149527 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:42.344489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1ddbf5-3bc7-4d88-911b-b5f72a149527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:42.402726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fd1ddbf5-3bc7-4d88-911b-b5f72a149527 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:42.404115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fd1ddbf5-3bc7-4d88-911b-b5f72a149527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:42.509420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 678/1359] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 20:34:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:34:42.510640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e9757-57b6-471d-9ed9-5977a75e5b77 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:42.525537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e9757-57b6-471d-9ed9-5977a75e5b77 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:42.532841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d38606-4bd6-4c04-ad2d-b9ca16efb2db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:42.533589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d38606-4bd6-4c04-ad2d-b9ca16efb2db service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:42.537186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d38606-4bd6-4c04-ad2d-b9ca16efb2db service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:42.537186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d38606-4bd6-4c04-ad2d-b9ca16efb2db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:42.543089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d38606-4bd6-4c04-ad2d-b9ca16efb2db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:42.545343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d38606-4bd6-4c04-ad2d-b9ca16efb2db service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:42.563743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 682/1360] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:34:45.002514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c98b-fbd1-43fb-a2d2-e0f2cb217d7b None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:45.003071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c98b-fbd1-43fb-a2d2-e0f2cb217d7b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:45.003071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d39c98b-fbd1-43fb-a2d2-e0f2cb217d7b None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:45.004635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 679/1361] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:45.009563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ba3f-47f9-4b7d-a1a1-bbedcd24bf1e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:45.009961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ba3f-47f9-4b7d-a1a1-bbedcd24bf1e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:45.010109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ba3f-47f9-4b7d-a1a1-bbedcd24bf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:45.028894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d7b7ba3f-47f9-4b7d-a1a1-bbedcd24bf1e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:45.030058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d7b7ba3f-47f9-4b7d-a1a1-bbedcd24bf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:45.038619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 683/1362] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:45.082921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39c98b-fbd1-43fb-a2d2-e0f2cb217d7b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:45.097269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d39c98b-fbd1-43fb-a2d2-e0f2cb217d7b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:45.102659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ecd188-56bf-41ea-9748-45d2c8d6536b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:45.103290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecd188-56bf-41ea-9748-45d2c8d6536b service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:45.103573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecd188-56bf-41ea-9748-45d2c8d6536b service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:45.103879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ecd188-56bf-41ea-9748-45d2c8d6536b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:45.112815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecd188-56bf-41ea-9748-45d2c8d6536b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:45.114379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ecd188-56bf-41ea-9748-45d2c8d6536b service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:45.124769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 680/1363] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:47.854362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b9ef-aaaf-46c7-b4a9-6f7b52820fb1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:47.855255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b9ef-aaaf-46c7-b4a9-6f7b52820fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:47.855537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e01b9ef-aaaf-46c7-b4a9-6f7b52820fb1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:47.857013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 684/1364] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:47.862833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4bc78-6daa-485c-a3e8-efec6f50ea66 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:47.863324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4bc78-6daa-485c-a3e8-efec6f50ea66 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:47.863408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4bc78-6daa-485c-a3e8-efec6f50ea66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:47.892744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-49a4bc78-6daa-485c-a3e8-efec6f50ea66 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:47.893964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-49a4bc78-6daa-485c-a3e8-efec6f50ea66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:47.902875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 681/1365] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:47.949437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dd90c-7210-4935-b9b5-19c938df57d9 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:47.949720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dd90c-7210-4935-b9b5-19c938df57d9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:47.950610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13dd90c-7210-4935-b9b5-19c938df57d9 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:47.951983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 685/1366] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 20:34:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:47.957191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48365-9bfb-4583-a7e3-f7ec218d4b0e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:47.957478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48365-9bfb-4583-a7e3-f7ec218d4b0e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:47.957753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb48365-9bfb-4583-a7e3-f7ec218d4b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:47.975174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7bb48365-9bfb-4583-a7e3-f7ec218d4b0e None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:47.976402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7bb48365-9bfb-4583-a7e3-f7ec218d4b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:47.985392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 682/1367] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 20:34:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:47.992922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dd90c-7210-4935-b9b5-19c938df57d9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:48.007059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13dd90c-7210-4935-b9b5-19c938df57d9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:48.012311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295cfdfe-c39a-4e6d-87dc-db797ff3bb4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:48.013590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cfdfe-c39a-4e6d-87dc-db797ff3bb4c service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:48.013864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cfdfe-c39a-4e6d-87dc-db797ff3bb4c service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:48.014129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295cfdfe-c39a-4e6d-87dc-db797ff3bb4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:48.022002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295cfdfe-c39a-4e6d-87dc-db797ff3bb4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:48.023546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295cfdfe-c39a-4e6d-87dc-db797ff3bb4c service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:48.033003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 686/1368] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:51.092628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200df141-8fe5-4396-b70c-6c814b537dd6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.093166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200df141-8fe5-4396-b70c-6c814b537dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.093166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200df141-8fe5-4396-b70c-6c814b537dd6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.094955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 683/1369] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:51.100166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b1d2-bc72-4f40-b967-f0c7937ed454 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.100590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b1d2-bc72-4f40-b967-f0c7937ed454 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.100715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315b1d2-bc72-4f40-b967-f0c7937ed454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.119269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e315b1d2-bc72-4f40-b967-f0c7937ed454 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:51.120486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e315b1d2-bc72-4f40-b967-f0c7937ed454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:51.129048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 687/1370] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:51.157651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e58b3-b382-4a2b-a317-15f2b2b92be7 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.157967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e58b3-b382-4a2b-a317-15f2b2b92be7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.158205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e58b3-b382-4a2b-a317-15f2b2b92be7 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.159695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 684/1371] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:51.164744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d253-305f-426d-9d50-ef9827b55197 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.165125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d253-305f-426d-9d50-ef9827b55197 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.165302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d253-305f-426d-9d50-ef9827b55197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.174955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e58b3-b382-4a2b-a317-15f2b2b92be7 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:51.187724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e3a8d253-305f-426d-9d50-ef9827b55197 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:51.188799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e58b3-b382-4a2b-a317-15f2b2b92be7 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:51.188916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e3a8d253-305f-426d-9d50-ef9827b55197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:51.194174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc440f-ea40-44b6-970d-3a08fc6e8345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:51.194808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc440f-ea40-44b6-970d-3a08fc6e8345 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.195084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc440f-ea40-44b6-970d-3a08fc6e8345 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.195358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc440f-ea40-44b6-970d-3a08fc6e8345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.197372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 688/1372] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:51.205344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc440f-ea40-44b6-970d-3a08fc6e8345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:51.207163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc440f-ea40-44b6-970d-3a08fc6e8345 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:51.217899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 685/1373] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:34:51.504066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1f8f7-6886-49e9-8da7-71eb29af126a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.504066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1f8f7-6886-49e9-8da7-71eb29af126a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.504066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1f8f7-6886-49e9-8da7-71eb29af126a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.506725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 689/1374] 10.222.0.41 () {54 vars in 963 bytes} [Mon Jun 6 20:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:51.511496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2b8b7-6b72-4ad7-9558-72a6409f0c91 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.511496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2b8b7-6b72-4ad7-9558-72a6409f0c91 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.511496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2b8b7-6b72-4ad7-9558-72a6409f0c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.561541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:34:51.565309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-08c2b8b7-6b72-4ad7-9558-72a6409f0c91 None None] MFA Rules not processed for user `9110e7fab9084faa94b5d345bb9d519a`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:51.566633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-08c2b8b7-6b72-4ad7-9558-72a6409f0c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:51.633852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 686/1375] 10.222.0.41 () {58 vars in 1060 bytes} [Mon Jun 6 20:34:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:34:51.642025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1f8f7-6886-49e9-8da7-71eb29af126a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:51.653249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1f8f7-6886-49e9-8da7-71eb29af126a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:51.656854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee598a-0c6b-4a7c-9696-6a5bcd981aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:51.658809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee598a-0c6b-4a7c-9696-6a5bcd981aec service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:51.658809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee598a-0c6b-4a7c-9696-6a5bcd981aec service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:51.658809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee598a-0c6b-4a7c-9696-6a5bcd981aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:51.668492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee598a-0c6b-4a7c-9696-6a5bcd981aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:51.670281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee598a-0c6b-4a7c-9696-6a5bcd981aec service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:51.681608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 690/1376] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:53.670455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586885-56eb-48b3-a712-54e8e01ae7b7 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:53.671034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586885-56eb-48b3-a712-54e8e01ae7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:53.671034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0586885-56eb-48b3-a712-54e8e01ae7b7 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:53.672621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 687/1377] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:53.678430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29b5f-bb8e-4fa0-b8bb-39518504a83e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:53.678959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29b5f-bb8e-4fa0-b8bb-39518504a83e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:53.678959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29b5f-bb8e-4fa0-b8bb-39518504a83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:53.698366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e4f29b5f-bb8e-4fa0-b8bb-39518504a83e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:53.699576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e4f29b5f-bb8e-4fa0-b8bb-39518504a83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:53.707519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 691/1378] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:53.753527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586885-56eb-48b3-a712-54e8e01ae7b7 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:53.768735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0586885-56eb-48b3-a712-54e8e01ae7b7 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:53.774012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ecf6c-e2bc-426c-bd76-1b6ac6f07823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:53.774641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ecf6c-e2bc-426c-bd76-1b6ac6f07823 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:53.774922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ecf6c-e2bc-426c-bd76-1b6ac6f07823 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:53.775195 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ecf6c-e2bc-426c-bd76-1b6ac6f07823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:53.784261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ecf6c-e2bc-426c-bd76-1b6ac6f07823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:53.785769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ecf6c-e2bc-426c-bd76-1b6ac6f07823 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:53.796273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 688/1379] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:56.191813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72065be-feb8-47a9-a539-e12c9c46a12c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:56.192811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72065be-feb8-47a9-a539-e12c9c46a12c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:56.193107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72065be-feb8-47a9-a539-e12c9c46a12c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:56.194701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 692/1380] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:56.200581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd294bc-f2d9-491b-9fa4-345a14d68783 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:56.201012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd294bc-f2d9-491b-9fa4-345a14d68783 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:56.201135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd294bc-f2d9-491b-9fa4-345a14d68783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:56.218897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1dd294bc-f2d9-491b-9fa4-345a14d68783 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:56.220127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1dd294bc-f2d9-491b-9fa4-345a14d68783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:56.232034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 689/1381] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:56.273282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72065be-feb8-47a9-a539-e12c9c46a12c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:56.288241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72065be-feb8-47a9-a539-e12c9c46a12c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:56.294064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b96291-873b-4ed6-b0e0-9f16d8565b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:56.294840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96291-873b-4ed6-b0e0-9f16d8565b11 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:56.295225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96291-873b-4ed6-b0e0-9f16d8565b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:56.295600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b96291-873b-4ed6-b0e0-9f16d8565b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:56.304017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b96291-873b-4ed6-b0e0-9f16d8565b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:56.305678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b96291-873b-4ed6-b0e0-9f16d8565b11 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:56.316076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 693/1382] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:34:58.698264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abde045-3ec0-4ac1-bbaf-48d7819a97cb None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:58.699131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abde045-3ec0-4ac1-bbaf-48d7819a97cb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:58.699588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abde045-3ec0-4ac1-bbaf-48d7819a97cb None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:58.704029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 690/1383] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:34:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:34:58.706961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e0532-e9be-4397-83b5-8e04d367d310 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:58.707471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e0532-e9be-4397-83b5-8e04d367d310 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:58.707787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e0532-e9be-4397-83b5-8e04d367d310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:58.726998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-de2e0532-e9be-4397-83b5-8e04d367d310 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:34:58.728359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-de2e0532-e9be-4397-83b5-8e04d367d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:34:58.737101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 694/1384] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:34:58.779784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abde045-3ec0-4ac1-bbaf-48d7819a97cb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:34:58.794758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abde045-3ec0-4ac1-bbaf-48d7819a97cb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:34:58.800529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2744e7-38fc-42e6-b476-4a77d12a31ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:34:58.801264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2744e7-38fc-42e6-b476-4a77d12a31ec service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:34:58.801646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2744e7-38fc-42e6-b476-4a77d12a31ec service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:34:58.802038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2744e7-38fc-42e6-b476-4a77d12a31ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:34:58.810938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2744e7-38fc-42e6-b476-4a77d12a31ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:34:58.812701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2744e7-38fc-42e6-b476-4a77d12a31ec service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:34:58.823102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 691/1385] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:01.282253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb7172-ebdd-432b-a17c-25467ac44d65 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:01.282900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb7172-ebdd-432b-a17c-25467ac44d65 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:01.283015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb7172-ebdd-432b-a17c-25467ac44d65 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:01.284598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 695/1386] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:01.289982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf4ba-5511-49e9-ad4c-900e9bf2b2b7 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:01.290529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf4ba-5511-49e9-ad4c-900e9bf2b2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:01.290529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbf4ba-5511-49e9-ad4c-900e9bf2b2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:01.309850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f3dbf4ba-5511-49e9-ad4c-900e9bf2b2b7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:01.311017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f3dbf4ba-5511-49e9-ad4c-900e9bf2b2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:01.318946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 692/1387] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:01.364188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb7172-ebdd-432b-a17c-25467ac44d65 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:01.378617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb7172-ebdd-432b-a17c-25467ac44d65 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:01.385374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c36fbc-625b-412d-8be3-89c74bb5b7c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:01.386113 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c36fbc-625b-412d-8be3-89c74bb5b7c0 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:01.386450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c36fbc-625b-412d-8be3-89c74bb5b7c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:01.386771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c36fbc-625b-412d-8be3-89c74bb5b7c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:01.397024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c36fbc-625b-412d-8be3-89c74bb5b7c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:01.398946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c36fbc-625b-412d-8be3-89c74bb5b7c0 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:01.411994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 696/1388] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:03.877694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077cf9-162b-4625-a6f7-f92bf049a97b None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:03.878225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077cf9-162b-4625-a6f7-f92bf049a97b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:03.878377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41077cf9-162b-4625-a6f7-f92bf049a97b None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:03.880075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 693/1389] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:35:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:03.885337 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faf60f-a8c0-4ac7-af99-0f696487dca2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:03.885854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faf60f-a8c0-4ac7-af99-0f696487dca2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:03.885962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26faf60f-a8c0-4ac7-af99-0f696487dca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:03.904363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-26faf60f-a8c0-4ac7-af99-0f696487dca2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:03.905578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-26faf60f-a8c0-4ac7-af99-0f696487dca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:03.913844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 697/1390] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:03.958513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41077cf9-162b-4625-a6f7-f92bf049a97b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:03.975785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41077cf9-162b-4625-a6f7-f92bf049a97b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:03.983376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a43ce4-bcd8-49c0-8768-670c834f381f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:03.984048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a43ce4-bcd8-49c0-8768-670c834f381f service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:03.984333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a43ce4-bcd8-49c0-8768-670c834f381f service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:03.984651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a43ce4-bcd8-49c0-8768-670c834f381f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:03.994832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a43ce4-bcd8-49c0-8768-670c834f381f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:03.997475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a43ce4-bcd8-49c0-8768-670c834f381f service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:04.009443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 694/1391] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:35:06.698989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ac1f5-3de9-4860-952e-51489c0f9f1d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:06.699600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ac1f5-3de9-4860-952e-51489c0f9f1d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:06.699600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ac1f5-3de9-4860-952e-51489c0f9f1d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:06.701430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 698/1392] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:35:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:06.706624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b80d2-bb3d-48cb-898e-77f4ec79ebf4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:06.707102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b80d2-bb3d-48cb-898e-77f4ec79ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:06.707198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b80d2-bb3d-48cb-898e-77f4ec79ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:06.724472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1a7b80d2-bb3d-48cb-898e-77f4ec79ebf4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:06.725653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1a7b80d2-bb3d-48cb-898e-77f4ec79ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:06.733341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 695/1393] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:06.777115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ac1f5-3de9-4860-952e-51489c0f9f1d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:06.791994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ac1f5-3de9-4860-952e-51489c0f9f1d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:06.797699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d2640-e436-4f92-9d2a-a1eac8eb5484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:06.798349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d2640-e436-4f92-9d2a-a1eac8eb5484 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:06.798634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d2640-e436-4f92-9d2a-a1eac8eb5484 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:06.798914 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d2640-e436-4f92-9d2a-a1eac8eb5484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:06.807748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d2640-e436-4f92-9d2a-a1eac8eb5484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:06.809830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d2640-e436-4f92-9d2a-a1eac8eb5484 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:06.819110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 699/1394] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:07.928474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635b16e-5e42-4af0-81a6-c22e52c672c6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:07.928969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635b16e-5e42-4af0-81a6-c22e52c672c6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:07.928969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635b16e-5e42-4af0-81a6-c22e52c672c6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:07.930364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 696/1395] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:07.936398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65a58e-dad6-4369-b18b-8f0f766bf1a5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:07.936836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65a58e-dad6-4369-b18b-8f0f766bf1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:07.936959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d65a58e-dad6-4369-b18b-8f0f766bf1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:07.942901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=55 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:35:07.955161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3d65a58e-dad6-4369-b18b-8f0f766bf1a5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:07.956388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3d65a58e-dad6-4369-b18b-8f0f766bf1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:07.964165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 700/1396] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:08.019579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92e598-9b4a-4ad5-b991-d32eb8cf8af6 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:08.019895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92e598-9b4a-4ad5-b991-d32eb8cf8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:08.020171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92e598-9b4a-4ad5-b991-d32eb8cf8af6 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:08.021530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 697/1397] 10.222.0.41 () {54 vars in 962 bytes} [Mon Jun 6 20:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:08.026911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6d21-f88b-4a4f-8d6f-00d518a94fd7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:08.027204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6d21-f88b-4a4f-8d6f-00d518a94fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:08.027483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cae6d21-f88b-4a4f-8d6f-00d518a94fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:08.077676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9cae6d21-f88b-4a4f-8d6f-00d518a94fd7 None None] MFA Rules not processed for user `bb84e744681149aea78fcc984605bf33`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:08.078903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9cae6d21-f88b-4a4f-8d6f-00d518a94fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:08.140209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 701/1398] 10.222.0.41 () {58 vars in 1059 bytes} [Mon Jun 6 20:35:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:08.147855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92e598-9b4a-4ad5-b991-d32eb8cf8af6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:08.159357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92e598-9b4a-4ad5-b991-d32eb8cf8af6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:08.162937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578cab7-1133-4d9f-8e59-37937144a577 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:08.163552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578cab7-1133-4d9f-8e59-37937144a577 service glance] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:08.163870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578cab7-1133-4d9f-8e59-37937144a577 service glance] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:08.164148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578cab7-1133-4d9f-8e59-37937144a577 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:08.173962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578cab7-1133-4d9f-8e59-37937144a577 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:08.175385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5578cab7-1133-4d9f-8e59-37937144a577 service glance] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:08.184731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 698/1399] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:08.854690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72a0b-c89f-4913-888c-f8ef1d30e969 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:08.854984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72a0b-c89f-4913-888c-f8ef1d30e969 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:08.855267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b72a0b-c89f-4913-888c-f8ef1d30e969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:08.910047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-44b72a0b-c89f-4913-888c-f8ef1d30e969 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:08.911288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-44b72a0b-c89f-4913-888c-f8ef1d30e969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:08.974094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 702/1400] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:35:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:09.019118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001991b-6152-4993-ae57-c42cee0b442a None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:09.019613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001991b-6152-4993-ae57-c42cee0b442a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:09.019822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001991b-6152-4993-ae57-c42cee0b442a None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:09.021206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 699/1401] 10.222.0.41 () {54 vars in 942 bytes} [Mon Jun 6 20:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:35:09.027927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d5d8-5314-4c4b-a201-7c36e8f4ee80 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:09.028214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d5d8-5314-4c4b-a201-7c36e8f4ee80 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:09.028511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de0d5d8-5314-4c4b-a201-7c36e8f4ee80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:09.076115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2de0d5d8-5314-4c4b-a201-7c36e8f4ee80 None None] MFA Rules not processed for user `e168d14c0eac4ee2a5acdc23c39118d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:09.077294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2de0d5d8-5314-4c4b-a201-7c36e8f4ee80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:09.089849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:35:09.147731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 703/1402] 10.222.0.41 () {58 vars in 1039 bytes} [Mon Jun 6 20:35:09 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:09.157281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001991b-6152-4993-ae57-c42cee0b442a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:09.171373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001991b-6152-4993-ae57-c42cee0b442a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:09.176233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef10184-8194-4e66-8ded-695bf92964e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:09.176861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef10184-8194-4e66-8ded-695bf92964e7 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:09.177145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef10184-8194-4e66-8ded-695bf92964e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:09.177417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef10184-8194-4e66-8ded-695bf92964e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:09.186025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef10184-8194-4e66-8ded-695bf92964e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:09.187620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef10184-8194-4e66-8ded-695bf92964e7 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:09.196130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 700/1403] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:35:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:35:09.206073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929be7ef-5266-4ae5-9f13-3687a109dc9d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:09.206365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929be7ef-5266-4ae5-9f13-3687a109dc9d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:09.206644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929be7ef-5266-4ae5-9f13-3687a109dc9d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:09.208212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 704/1404] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:09.213389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263508b4-bbf5-46a4-ab4f-6ec92004fc5e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:09.213686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263508b4-bbf5-46a4-ab4f-6ec92004fc5e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:09.213968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263508b4-bbf5-46a4-ab4f-6ec92004fc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:09.232673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-263508b4-bbf5-46a4-ab4f-6ec92004fc5e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:09.233877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-263508b4-bbf5-46a4-ab4f-6ec92004fc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:09.242189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 701/1405] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:35:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:35:09.286097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929be7ef-5266-4ae5-9f13-3687a109dc9d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:09.301179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929be7ef-5266-4ae5-9f13-3687a109dc9d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:09.306631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aae8ff-22fe-4e6c-b07b-25c58c89a183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:09.307254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aae8ff-22fe-4e6c-b07b-25c58c89a183 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:09.307535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aae8ff-22fe-4e6c-b07b-25c58c89a183 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:09.307851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aae8ff-22fe-4e6c-b07b-25c58c89a183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:09.316410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aae8ff-22fe-4e6c-b07b-25c58c89a183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:09.317966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aae8ff-22fe-4e6c-b07b-25c58c89a183 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:09.327165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 705/1406] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:11.722864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4511e3d-70bd-4d3d-88b2-90661bbe6530 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:11.723429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4511e3d-70bd-4d3d-88b2-90661bbe6530 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:11.723429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4511e3d-70bd-4d3d-88b2-90661bbe6530 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:11.727707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 702/1407] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:11.733849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5b72-f312-4f17-9a91-4d828542459a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:11.733849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5b72-f312-4f17-9a91-4d828542459a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:11.733849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81c5b72-f312-4f17-9a91-4d828542459a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:11.751400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c81c5b72-f312-4f17-9a91-4d828542459a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:11.755571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c81c5b72-f312-4f17-9a91-4d828542459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:11.760472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 706/1408] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:11.804686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4511e3d-70bd-4d3d-88b2-90661bbe6530 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:11.819503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4511e3d-70bd-4d3d-88b2-90661bbe6530 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:11.825490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50cccdf-151d-4ff8-a2f6-47692000390e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:11.826181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cccdf-151d-4ff8-a2f6-47692000390e service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:11.826475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cccdf-151d-4ff8-a2f6-47692000390e service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:11.826754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50cccdf-151d-4ff8-a2f6-47692000390e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:11.835860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cccdf-151d-4ff8-a2f6-47692000390e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:11.837355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50cccdf-151d-4ff8-a2f6-47692000390e service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:11.848555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 703/1409] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:21.745370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=33 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:35:21.843603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=72 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:35:33.161461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdbda4-57a4-423a-9344-960923603f4c None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:33.162011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdbda4-57a4-423a-9344-960923603f4c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:33.162011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fdbda4-57a4-423a-9344-960923603f4c None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:33.163408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 707/1410] 10.222.0.41 () {54 vars in 918 bytes} [Mon Jun 6 20:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:33.168347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb142-762d-4049-8071-7d022ec32239 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:33.168793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb142-762d-4049-8071-7d022ec32239 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:33.168906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81adb142-762d-4049-8071-7d022ec32239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:33.186511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-81adb142-762d-4049-8071-7d022ec32239 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:33.187740 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-81adb142-762d-4049-8071-7d022ec32239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:33.196038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 704/1411] 10.222.0.41 () {58 vars in 1015 bytes} [Mon Jun 6 20:35:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:33.210455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11830bba-b530-42bb-ac8a-bed28556a680 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:33.210769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11830bba-b530-42bb-ac8a-bed28556a680 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:33.211048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11830bba-b530-42bb-ac8a-bed28556a680 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:33.212423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 708/1412] 10.222.0.41 () {54 vars in 918 bytes} [Mon Jun 6 20:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:33.217422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9afb98-9bba-4124-a7c3-3ec257043e06 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:33.217737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9afb98-9bba-4124-a7c3-3ec257043e06 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:33.218020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9afb98-9bba-4124-a7c3-3ec257043e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:33.235792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5a9afb98-9bba-4124-a7c3-3ec257043e06 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:33.236973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5a9afb98-9bba-4124-a7c3-3ec257043e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:33.245938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 705/1413] 10.222.0.41 () {58 vars in 1015 bytes} [Mon Jun 6 20:35:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:35:35.934643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859349c-fe98-40fa-8c00-6e896df275e7 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:35.936767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859349c-fe98-40fa-8c00-6e896df275e7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:35.936767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859349c-fe98-40fa-8c00-6e896df275e7 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:35.937125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:35.942396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89205abd-b64b-4172-943d-09bbc668c5e2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:35.942831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89205abd-b64b-4172-943d-09bbc668c5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:35.942936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89205abd-b64b-4172-943d-09bbc668c5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:35.960873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-89205abd-b64b-4172-943d-09bbc668c5e2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:35.962066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-89205abd-b64b-4172-943d-09bbc668c5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:35.969741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:36.012296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859349c-fe98-40fa-8c00-6e896df275e7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:36.027066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859349c-fe98-40fa-8c00-6e896df275e7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:36.032429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98346af9-abec-462a-976d-d3661d3d3f29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:36.033051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346af9-abec-462a-976d-d3661d3d3f29 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:36.033340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346af9-abec-462a-976d-d3661d3d3f29 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:36.033614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98346af9-abec-462a-976d-d3661d3d3f29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:36.041557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98346af9-abec-462a-976d-d3661d3d3f29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:36.043116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98346af9-abec-462a-976d-d3661d3d3f29 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:36.052391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 710/1416] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:37.012555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afbdb4-fd9b-4dab-8050-7f6859923fa0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:37.013126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afbdb4-fd9b-4dab-8050-7f6859923fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:37.013126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afbdb4-fd9b-4dab-8050-7f6859923fa0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:37.014998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:37.019805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d1111-d92c-4d0b-9e76-fd9a21adc773 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:37.020260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d1111-d92c-4d0b-9e76-fd9a21adc773 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:37.020367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d1111-d92c-4d0b-9e76-fd9a21adc773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:37.038589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d01d1111-d92c-4d0b-9e76-fd9a21adc773 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:37.039799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d01d1111-d92c-4d0b-9e76-fd9a21adc773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:37.047485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:37.204404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afbdb4-fd9b-4dab-8050-7f6859923fa0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:37.218777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afbdb4-fd9b-4dab-8050-7f6859923fa0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:37.224057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c3ab-f5db-4d34-957c-acd5eeb11723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:37.224687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c3ab-f5db-4d34-957c-acd5eeb11723 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:37.224971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c3ab-f5db-4d34-957c-acd5eeb11723 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:37.225245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c3ab-f5db-4d34-957c-acd5eeb11723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:37.234134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8c3ab-f5db-4d34-957c-acd5eeb11723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:37.235803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db8c3ab-f5db-4d34-957c-acd5eeb11723 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:37.245292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 708/1419] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:38.169862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ba9fe-71de-415f-8ff8-74b10f96d753 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:38.170493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ba9fe-71de-415f-8ff8-74b10f96d753 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:38.170493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ba9fe-71de-415f-8ff8-74b10f96d753 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:38.171788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:38.176812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1125c9-d1f8-4beb-99d7-b3ec074bc660 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:38.177228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1125c9-d1f8-4beb-99d7-b3ec074bc660 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:38.177354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1125c9-d1f8-4beb-99d7-b3ec074bc660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:38.195655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fd1125c9-d1f8-4beb-99d7-b3ec074bc660 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:38.196877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fd1125c9-d1f8-4beb-99d7-b3ec074bc660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:38.205487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:38.356702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ba9fe-71de-415f-8ff8-74b10f96d753 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:38.370619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ba9fe-71de-415f-8ff8-74b10f96d753 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:38.376135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7686310-fec3-4b85-a01c-ba0b4d00fb3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:38.376767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7686310-fec3-4b85-a01c-ba0b4d00fb3c service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:38.377049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7686310-fec3-4b85-a01c-ba0b4d00fb3c service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:38.377324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7686310-fec3-4b85-a01c-ba0b4d00fb3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:38.385486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7686310-fec3-4b85-a01c-ba0b4d00fb3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:38.387067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7686310-fec3-4b85-a01c-ba0b4d00fb3c service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:38.396826 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 713/1422] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:39.289475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73c4e-a10f-447f-8eec-b756c23556ea None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:39.289988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73c4e-a10f-447f-8eec-b756c23556ea None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:39.290126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a73c4e-a10f-447f-8eec-b756c23556ea None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:39.291395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:39.296680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea318cb-0624-45d5-81f9-aae5513d63fc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:39.297116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea318cb-0624-45d5-81f9-aae5513d63fc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:39.297238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea318cb-0624-45d5-81f9-aae5513d63fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:39.315624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3ea318cb-0624-45d5-81f9-aae5513d63fc None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:39.316832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3ea318cb-0624-45d5-81f9-aae5513d63fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:39.324444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:39.471424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73c4e-a10f-447f-8eec-b756c23556ea None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:39.486325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a73c4e-a10f-447f-8eec-b756c23556ea None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:39.491546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb08e7-a553-4cc7-a489-9a21981a090a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:39.492201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb08e7-a553-4cc7-a489-9a21981a090a service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:39.492486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb08e7-a553-4cc7-a489-9a21981a090a service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:39.492763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb08e7-a553-4cc7-a489-9a21981a090a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:39.500885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb08e7-a553-4cc7-a489-9a21981a090a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:39.502486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb08e7-a553-4cc7-a489-9a21981a090a service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:39.511842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 711/1425] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:40.398388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226119f5-7662-496e-acad-725c37d0a4c1 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:40.398945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226119f5-7662-496e-acad-725c37d0a4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:40.399037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226119f5-7662-496e-acad-725c37d0a4c1 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:40.400634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:40.406086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6383-4937-47eb-a94d-58c670e32a7f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:40.406511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6383-4937-47eb-a94d-58c670e32a7f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:40.406622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f6383-4937-47eb-a94d-58c670e32a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:40.424773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-121f6383-4937-47eb-a94d-58c670e32a7f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:40.425956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-121f6383-4937-47eb-a94d-58c670e32a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:40.433932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:40.583267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226119f5-7662-496e-acad-725c37d0a4c1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:40.597637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226119f5-7662-496e-acad-725c37d0a4c1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:40.603057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c08ba-c5b9-4ea7-b515-014d0b434e0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:40.603711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c08ba-c5b9-4ea7-b515-014d0b434e0f service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:40.603996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c08ba-c5b9-4ea7-b515-014d0b434e0f service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:40.604291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c08ba-c5b9-4ea7-b515-014d0b434e0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:40.612224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c08ba-c5b9-4ea7-b515-014d0b434e0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:40.613764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2c08ba-c5b9-4ea7-b515-014d0b434e0f service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:40.622833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 716/1428] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:41.525902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d51b77-1871-466a-96f7-6c4e1d776c51 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:41.526414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d51b77-1871-466a-96f7-6c4e1d776c51 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:41.526414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d51b77-1871-466a-96f7-6c4e1d776c51 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:41.527760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:41.532702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54423c4-d6e9-4fe5-bc19-06d8b4ac9ce8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:41.533112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54423c4-d6e9-4fe5-bc19-06d8b4ac9ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:41.533251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54423c4-d6e9-4fe5-bc19-06d8b4ac9ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:41.551564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a54423c4-d6e9-4fe5-bc19-06d8b4ac9ce8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:41.552843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a54423c4-d6e9-4fe5-bc19-06d8b4ac9ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:41.561257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:41.708282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d51b77-1871-466a-96f7-6c4e1d776c51 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:41.722720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d51b77-1871-466a-96f7-6c4e1d776c51 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:41.728039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bde05-fa59-405c-9d9a-959b81e177c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:41.728666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bde05-fa59-405c-9d9a-959b81e177c3 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:41.728947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bde05-fa59-405c-9d9a-959b81e177c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:41.729221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bde05-fa59-405c-9d9a-959b81e177c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:41.737881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bde05-fa59-405c-9d9a-959b81e177c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:41.739437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bde05-fa59-405c-9d9a-959b81e177c3 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:41.748707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 714/1431] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:42.686873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6aaaa-844c-4db1-8ce7-a8db040ea101 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:42.687412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6aaaa-844c-4db1-8ce7-a8db040ea101 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:42.687412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c6aaaa-844c-4db1-8ce7-a8db040ea101 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:42.688712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:42.693486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb5e8-1a56-46ce-bdc4-f65e3c33a0ef None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:42.693919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb5e8-1a56-46ce-bdc4-f65e3c33a0ef None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:42.694027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fb5e8-1a56-46ce-bdc4-f65e3c33a0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:42.712456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4b0fb5e8-1a56-46ce-bdc4-f65e3c33a0ef None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:42.713636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4b0fb5e8-1a56-46ce-bdc4-f65e3c33a0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:42.721749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:42.869678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6aaaa-844c-4db1-8ce7-a8db040ea101 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:42.884196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c6aaaa-844c-4db1-8ce7-a8db040ea101 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:42.890006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed5b4a-c378-4b27-b773-deb203c529ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:42.890629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed5b4a-c378-4b27-b773-deb203c529ea service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:42.890907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed5b4a-c378-4b27-b773-deb203c529ea service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:42.891178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed5b4a-c378-4b27-b773-deb203c529ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:42.899057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed5b4a-c378-4b27-b773-deb203c529ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:42.900904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed5b4a-c378-4b27-b773-deb203c529ea service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:42.910560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 719/1434] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:43.821359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5aa56-94e5-4f70-b85f-c5588566cb27 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:43.821933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5aa56-94e5-4f70-b85f-c5588566cb27 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:43.821933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5aa56-94e5-4f70-b85f-c5588566cb27 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:43.823264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:43.828229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8fe9e-079d-4813-9fc2-658fbebac341 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:43.828677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8fe9e-079d-4813-9fc2-658fbebac341 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:43.828777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae8fe9e-079d-4813-9fc2-658fbebac341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:43.846749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-aae8fe9e-079d-4813-9fc2-658fbebac341 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:43.847978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aae8fe9e-079d-4813-9fc2-658fbebac341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:43.855851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:44.004178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5aa56-94e5-4f70-b85f-c5588566cb27 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:44.018342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5aa56-94e5-4f70-b85f-c5588566cb27 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:44.023690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07453522-481e-4bca-a61d-f486e5692fc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:44.024330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07453522-481e-4bca-a61d-f486e5692fc2 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:44.024615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07453522-481e-4bca-a61d-f486e5692fc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:44.024925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07453522-481e-4bca-a61d-f486e5692fc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:44.033677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07453522-481e-4bca-a61d-f486e5692fc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:44.035248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07453522-481e-4bca-a61d-f486e5692fc2 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:44.045124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 717/1437] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:44.950295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc6344-1adf-4f07-8303-51691244e4e6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:44.950833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc6344-1adf-4f07-8303-51691244e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:44.950953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bc6344-1adf-4f07-8303-51691244e4e6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:44.952284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:44.957422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8fe52-3f91-4319-80ba-7e08e605e38e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:44.957860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8fe52-3f91-4319-80ba-7e08e605e38e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:44.957970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c8fe52-3f91-4319-80ba-7e08e605e38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:44.976083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-e4c8fe52-3f91-4319-80ba-7e08e605e38e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:44.977272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e4c8fe52-3f91-4319-80ba-7e08e605e38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:44.985613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:45.134106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc6344-1adf-4f07-8303-51691244e4e6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:45.148638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bc6344-1adf-4f07-8303-51691244e4e6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:45.154239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90a6ec-146b-4910-94e5-9edb71e2972c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:45.155454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90a6ec-146b-4910-94e5-9edb71e2972c service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:45.155767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90a6ec-146b-4910-94e5-9edb71e2972c service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:45.156047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90a6ec-146b-4910-94e5-9edb71e2972c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:45.163979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90a6ec-146b-4910-94e5-9edb71e2972c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:45.165555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90a6ec-146b-4910-94e5-9edb71e2972c service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:45.174784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 722/1440] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:47.272120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b2c8f-297e-4b46-8e13-df87b05a6f4b None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:47.272618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b2c8f-297e-4b46-8e13-df87b05a6f4b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:47.272731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b2c8f-297e-4b46-8e13-df87b05a6f4b None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:47.274073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:47.279533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca612-28e8-472b-85a2-8bffb1201564 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:47.279994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca612-28e8-472b-85a2-8bffb1201564 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:47.280145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ca612-28e8-472b-85a2-8bffb1201564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:47.297948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-721ca612-28e8-472b-85a2-8bffb1201564 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:47.299136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-721ca612-28e8-472b-85a2-8bffb1201564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:47.306875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:47.345150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b2c8f-297e-4b46-8e13-df87b05a6f4b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:47.355614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b2c8f-297e-4b46-8e13-df87b05a6f4b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:47.359617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9d13f-4f6d-4c98-bda3-3f97abb90669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:47.360269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d13f-4f6d-4c98-bda3-3f97abb90669 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:47.360560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d13f-4f6d-4c98-bda3-3f97abb90669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:47.360836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9d13f-4f6d-4c98-bda3-3f97abb90669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:47.369474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9d13f-4f6d-4c98-bda3-3f97abb90669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:47.370889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9d13f-4f6d-4c98-bda3-3f97abb90669 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:47.380669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 720/1443] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:49.737505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f765c-0d39-43e8-91a9-0af799058112 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:49.748960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f765c-0d39-43e8-91a9-0af799058112 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:49.748960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f765c-0d39-43e8-91a9-0af799058112 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:49.748960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:49.749264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58193-cd28-48b8-bcf3-a759438442e1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:49.749264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58193-cd28-48b8-bcf3-a759438442e1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:49.749264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58193-cd28-48b8-bcf3-a759438442e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:49.763326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-eeb58193-cd28-48b8-bcf3-a759438442e1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:49.765075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-eeb58193-cd28-48b8-bcf3-a759438442e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:49.773026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:49.811634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f765c-0d39-43e8-91a9-0af799058112 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:49.822179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f765c-0d39-43e8-91a9-0af799058112 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:49.825632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362efb68-3e50-4d6d-8e89-9632585dc2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:49.826249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362efb68-3e50-4d6d-8e89-9632585dc2de service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:49.827039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362efb68-3e50-4d6d-8e89-9632585dc2de service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:49.827309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362efb68-3e50-4d6d-8e89-9632585dc2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:49.836003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362efb68-3e50-4d6d-8e89-9632585dc2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:49.837405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362efb68-3e50-4d6d-8e89-9632585dc2de service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:49.846898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 725/1446] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:52.427831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaa8ab-b188-4a97-8cf8-c303be8bd65a None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:52.428466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaa8ab-b188-4a97-8cf8-c303be8bd65a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:52.428581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbaa8ab-b188-4a97-8cf8-c303be8bd65a None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:52.430089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 06 20:35:52.436736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c936-186f-4624-a7bb-a3a60a1ed5c4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:52.437176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c936-186f-4624-a7bb-a3a60a1ed5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:52.437359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345c936-186f-4624-a7bb-a3a60a1ed5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:52.457894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a345c936-186f-4624-a7bb-a3a60a1ed5c4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:52.459097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a345c936-186f-4624-a7bb-a3a60a1ed5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:52.467296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:52.506504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbaa8ab-b188-4a97-8cf8-c303be8bd65a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:52.517805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbaa8ab-b188-4a97-8cf8-c303be8bd65a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:52.521526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c66593-bbbf-488d-8f0b-df10271fe1e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:52.522184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66593-bbbf-488d-8f0b-df10271fe1e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:52.522470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66593-bbbf-488d-8f0b-df10271fe1e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:52.522748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c66593-bbbf-488d-8f0b-df10271fe1e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:52.532177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66593-bbbf-488d-8f0b-df10271fe1e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:52.533597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c66593-bbbf-488d-8f0b-df10271fe1e5 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:52.542865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 723/1449] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:54.898586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872dcb8-a248-4758-b9c5-78cb4559899f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:54.899166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872dcb8-a248-4758-b9c5-78cb4559899f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:54.899166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872dcb8-a248-4758-b9c5-78cb4559899f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:54.900778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:54.906092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efde51-b059-406f-8ebd-18a40acc5eb8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:54.906509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efde51-b059-406f-8ebd-18a40acc5eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:54.906625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efde51-b059-406f-8ebd-18a40acc5eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:54.924564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-23efde51-b059-406f-8ebd-18a40acc5eb8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:54.925747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-23efde51-b059-406f-8ebd-18a40acc5eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:54.933364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:54.971535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872dcb8-a248-4758-b9c5-78cb4559899f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:54.982003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872dcb8-a248-4758-b9c5-78cb4559899f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:54.986198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d18af-dfa7-4f97-9403-eef8ba29926c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:54.986823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d18af-dfa7-4f97-9403-eef8ba29926c service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:54.987102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d18af-dfa7-4f97-9403-eef8ba29926c service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:54.987374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d18af-dfa7-4f97-9403-eef8ba29926c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:54.996149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d18af-dfa7-4f97-9403-eef8ba29926c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:54.997584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d18af-dfa7-4f97-9403-eef8ba29926c service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:55.007314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 728/1452] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:35:57.283840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8f9a-deb5-4b91-b443-e0835fa9ac6d None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:57.284353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8f9a-deb5-4b91-b443-e0835fa9ac6d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:57.284481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8f9a-deb5-4b91-b443-e0835fa9ac6d None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:57.285794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:57.291071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63384694-8cd3-46b1-8d8a-cc19b51a4aeb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:57.291508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63384694-8cd3-46b1-8d8a-cc19b51a4aeb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:57.291614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63384694-8cd3-46b1-8d8a-cc19b51a4aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:57.308701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-63384694-8cd3-46b1-8d8a-cc19b51a4aeb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:57.309881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-63384694-8cd3-46b1-8d8a-cc19b51a4aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:57.317516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:57.355580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f8f9a-deb5-4b91-b443-e0835fa9ac6d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:57.366585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f8f9a-deb5-4b91-b443-e0835fa9ac6d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:57.370031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71357c27-787a-46a8-a870-d208d8bf34b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:57.370656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71357c27-787a-46a8-a870-d208d8bf34b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:57.370938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71357c27-787a-46a8-a870-d208d8bf34b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:57.371218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71357c27-787a-46a8-a870-d208d8bf34b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:57.380520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71357c27-787a-46a8-a870-d208d8bf34b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:57.382040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71357c27-787a-46a8-a870-d208d8bf34b7 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:57.391357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 726/1455] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:35:59.653346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f002b-ed4a-4889-ab9b-8e5737e768a0 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:59.653876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f002b-ed4a-4889-ab9b-8e5737e768a0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:59.653876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f002b-ed4a-4889-ab9b-8e5737e768a0 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:59.655777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:35:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:35:59.661095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec924cb-61cf-4cd7-a8ba-b9b00ab13f27 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:59.661497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec924cb-61cf-4cd7-a8ba-b9b00ab13f27 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:59.661660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec924cb-61cf-4cd7-a8ba-b9b00ab13f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:59.680343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-cec924cb-61cf-4cd7-a8ba-b9b00ab13f27 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:35:59.681538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cec924cb-61cf-4cd7-a8ba-b9b00ab13f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:35:59.689308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:35:59.726574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f002b-ed4a-4889-ab9b-8e5737e768a0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:35:59.737698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f002b-ed4a-4889-ab9b-8e5737e768a0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:35:59.741136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25453947-7055-4df8-adbd-447d4bd53c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:35:59.745720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453947-7055-4df8-adbd-447d4bd53c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:35:59.745720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453947-7055-4df8-adbd-447d4bd53c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:35:59.745720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453947-7055-4df8-adbd-447d4bd53c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:35:59.750865 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25453947-7055-4df8-adbd-447d4bd53c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:35:59.752301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25453947-7055-4df8-adbd-447d4bd53c18 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:35:59.761434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 731/1458] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:02.040752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f751e17-5b13-4b48-ba7b-b179db03c386 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:02.041298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f751e17-5b13-4b48-ba7b-b179db03c386 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:02.041298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f751e17-5b13-4b48-ba7b-b179db03c386 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:02.042673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:36:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:02.048685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0fa64-af06-409a-8996-af4e1c07af2d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:02.049108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0fa64-af06-409a-8996-af4e1c07af2d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:02.049264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0fa64-af06-409a-8996-af4e1c07af2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:02.055243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=70 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:36:02.066787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-44d0fa64-af06-409a-8996-af4e1c07af2d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:02.070260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-44d0fa64-af06-409a-8996-af4e1c07af2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:02.075526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:36:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:02.113627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f751e17-5b13-4b48-ba7b-b179db03c386 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:02.115283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4f751e17-5b13-4b48-ba7b-b179db03c386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:02.122701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=73 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:36:02.128398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f751e17-5b13-4b48-ba7b-b179db03c386 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:02.133252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f5e72-c37f-482c-85e9-57f59bc6c041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:02.133252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f5e72-c37f-482c-85e9-57f59bc6c041 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:02.133252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f5e72-c37f-482c-85e9-57f59bc6c041 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:02.133252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f5e72-c37f-482c-85e9-57f59bc6c041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:02.141977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f5e72-c37f-482c-85e9-57f59bc6c041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:02.143815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f5e72-c37f-482c-85e9-57f59bc6c041 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:02.153735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 729/1461] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:04.419945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a72c-133c-46e9-9e3a-83455d039c5f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:04.420502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a72c-133c-46e9-9e3a-83455d039c5f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:04.420502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46a72c-133c-46e9-9e3a-83455d039c5f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:04.421832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:04.427055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0771b-9d26-4cea-9241-bd9605f0fe5e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:04.427482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0771b-9d26-4cea-9241-bd9605f0fe5e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:04.427626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0771b-9d26-4cea-9241-bd9605f0fe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:04.445468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f0c0771b-9d26-4cea-9241-bd9605f0fe5e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:04.446656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f0c0771b-9d26-4cea-9241-bd9605f0fe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:04.454942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:04.490663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a72c-133c-46e9-9e3a-83455d039c5f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:04.501610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46a72c-133c-46e9-9e3a-83455d039c5f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:04.505024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573e0c-bed4-4eeb-a1c0-9b8931e3cb3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:04.505669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573e0c-bed4-4eeb-a1c0-9b8931e3cb3e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:04.505949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573e0c-bed4-4eeb-a1c0-9b8931e3cb3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:04.506223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573e0c-bed4-4eeb-a1c0-9b8931e3cb3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:04.515492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573e0c-bed4-4eeb-a1c0-9b8931e3cb3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:04.516934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573e0c-bed4-4eeb-a1c0-9b8931e3cb3e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:04.526289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 734/1464] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:06.730248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ec1fe-3286-4491-b494-1ed65aae5f3c None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:06.730792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ec1fe-3286-4491-b494-1ed65aae5f3c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:06.730792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ec1fe-3286-4491-b494-1ed65aae5f3c None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:06.732827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:36:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:06.737979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73d6c9-79f2-4eb8-bd2d-04ae460d5fcb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:06.738386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73d6c9-79f2-4eb8-bd2d-04ae460d5fcb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:06.738505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73d6c9-79f2-4eb8-bd2d-04ae460d5fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:06.755795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fa73d6c9-79f2-4eb8-bd2d-04ae460d5fcb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:06.756962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fa73d6c9-79f2-4eb8-bd2d-04ae460d5fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:06.764496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:06.801249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ec1fe-3286-4491-b494-1ed65aae5f3c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:06.811774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ec1fe-3286-4491-b494-1ed65aae5f3c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:06.815217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daea0c2-6ff0-4077-948d-24e15e2d75ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:06.815875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea0c2-6ff0-4077-948d-24e15e2d75ed service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:06.816162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea0c2-6ff0-4077-948d-24e15e2d75ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:06.816434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daea0c2-6ff0-4077-948d-24e15e2d75ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:06.825116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daea0c2-6ff0-4077-948d-24e15e2d75ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:06.827069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daea0c2-6ff0-4077-948d-24e15e2d75ed service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:06.836250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 732/1467] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:36:09.079903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b87d6-f802-42bc-b8d3-9ac0956b7356 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:09.080480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b87d6-f802-42bc-b8d3-9ac0956b7356 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:09.080480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b87d6-f802-42bc-b8d3-9ac0956b7356 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:09.081846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:09.087347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0fec8-01d7-4fc4-92dd-ed56becffd7c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:09.087772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0fec8-01d7-4fc4-92dd-ed56becffd7c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:09.087955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0fec8-01d7-4fc4-92dd-ed56becffd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:09.105547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-37c0fec8-01d7-4fc4-92dd-ed56becffd7c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:09.106760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-37c0fec8-01d7-4fc4-92dd-ed56becffd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:09.115111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:09.151886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b87d6-f802-42bc-b8d3-9ac0956b7356 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:09.162689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b87d6-f802-42bc-b8d3-9ac0956b7356 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:09.166446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c77cc3-df5d-4138-9fb2-725c6387484f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:09.167077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77cc3-df5d-4138-9fb2-725c6387484f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:09.167357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77cc3-df5d-4138-9fb2-725c6387484f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:09.167631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c77cc3-df5d-4138-9fb2-725c6387484f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:09.177357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c77cc3-df5d-4138-9fb2-725c6387484f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:09.180180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c77cc3-df5d-4138-9fb2-725c6387484f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:09.190064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 737/1470] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:19.100407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=17 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:36:19.185274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=17 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:36:40.919019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfb25-08de-4a2e-be02-0f3d03701dac None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:40.919571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfb25-08de-4a2e-be02-0f3d03701dac None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:40.919571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbfb25-08de-4a2e-be02-0f3d03701dac None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:40.921067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 734/1471] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:40.926423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a188ae-0d29-48c2-9610-41be0ce92e1f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:40.926900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a188ae-0d29-48c2-9610-41be0ce92e1f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:40.927006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a188ae-0d29-48c2-9610-41be0ce92e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:40.945117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a2a188ae-0d29-48c2-9610-41be0ce92e1f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:40.946328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a2a188ae-0d29-48c2-9610-41be0ce92e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:40.955717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 738/1472] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:41.002823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013dad-4b71-49e2-b046-ca42a4b977f7 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:41.003114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013dad-4b71-49e2-b046-ca42a4b977f7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:41.003396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c013dad-4b71-49e2-b046-ca42a4b977f7 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:41.004792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 735/1473] 10.222.0.41 () {54 vars in 962 bytes} [Mon Jun 6 20:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:41.009556 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec3396-a091-489b-96e4-67161b903d7e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:41.009839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec3396-a091-489b-96e4-67161b903d7e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:41.010110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec3396-a091-489b-96e4-67161b903d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:41.028456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-87ec3396-a091-489b-96e4-67161b903d7e None None] MFA Rules not processed for user `bb84e744681149aea78fcc984605bf33`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:41.029622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-87ec3396-a091-489b-96e4-67161b903d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:41.036257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 739/1474] 10.222.0.41 () {58 vars in 1059 bytes} [Mon Jun 6 20:36:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:41.043361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013dad-4b71-49e2-b046-ca42a4b977f7 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:41.054125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c013dad-4b71-49e2-b046-ca42a4b977f7 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:41.057827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948b95ad-1d2d-4ec5-8664-c4851761f154 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:41.058451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b95ad-1d2d-4ec5-8664-c4851761f154 service glance] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:41.058757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b95ad-1d2d-4ec5-8664-c4851761f154 service glance] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:41.059016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948b95ad-1d2d-4ec5-8664-c4851761f154 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:41.068496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b95ad-1d2d-4ec5-8664-c4851761f154 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:41.069920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948b95ad-1d2d-4ec5-8664-c4851761f154 service glance] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:41.079770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 736/1475] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:43.384981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef99e-f3d4-46aa-a86e-9ba511ccc2cc None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:43.385547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef99e-f3d4-46aa-a86e-9ba511ccc2cc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:43.385547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ef99e-f3d4-46aa-a86e-9ba511ccc2cc None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:43.386919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 740/1476] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:43.392006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37f71e-ec12-498b-b1df-2a5cb6930d5b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:43.392450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37f71e-ec12-498b-b1df-2a5cb6930d5b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:43.392557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37f71e-ec12-498b-b1df-2a5cb6930d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:43.410640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3a37f71e-ec12-498b-b1df-2a5cb6930d5b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:43.411811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3a37f71e-ec12-498b-b1df-2a5cb6930d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:43.420094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 737/1477] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:43.443718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef99e-f3d4-46aa-a86e-9ba511ccc2cc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:43.456558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ef99e-f3d4-46aa-a86e-9ba511ccc2cc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:43.460913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:43.461547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:43.461832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:43.462109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:43.462834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:43.464606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:36:43.464606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:36:43.464606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:36:43.465740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464aa28-96dd-4e4e-9bfc-1f4d70ff337f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:43.484257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 741/1478] 10.222.0.41 () {56 vars in 1108 bytes} [Mon Jun 6 20:36:43 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:36:45.730675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73a25f-ca01-444f-b5b4-47f7aaa0e27d None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:45.731192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73a25f-ca01-444f-b5b4-47f7aaa0e27d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:45.731836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73a25f-ca01-444f-b5b4-47f7aaa0e27d None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:45.733221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 738/1479] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:45.738452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752b7e7-f7f5-49c9-9108-f06391ca87a4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:45.741616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752b7e7-f7f5-49c9-9108-f06391ca87a4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:45.741616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752b7e7-f7f5-49c9-9108-f06391ca87a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:45.757049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3752b7e7-f7f5-49c9-9108-f06391ca87a4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:45.773122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3752b7e7-f7f5-49c9-9108-f06391ca87a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:45.773122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 742/1480] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:45.810839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73a25f-ca01-444f-b5b4-47f7aaa0e27d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:45.824849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73a25f-ca01-444f-b5b4-47f7aaa0e27d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:45.830734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b551cf9-71cb-4c0f-94f4-4e57dce1bae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:45.831360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b551cf9-71cb-4c0f-94f4-4e57dce1bae7 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:45.831774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b551cf9-71cb-4c0f-94f4-4e57dce1bae7 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:45.831998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b551cf9-71cb-4c0f-94f4-4e57dce1bae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:45.840056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b551cf9-71cb-4c0f-94f4-4e57dce1bae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:45.842191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b551cf9-71cb-4c0f-94f4-4e57dce1bae7 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:45.851592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 739/1481] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:48.266862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34e4c-a3f8-4bf7-a3cf-418259520811 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:48.267379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34e4c-a3f8-4bf7-a3cf-418259520811 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:48.267504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc34e4c-a3f8-4bf7-a3cf-418259520811 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:48.268815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 743/1482] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:48.274389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601bf5a-51f4-45bc-a3e1-f4f044c0eb31 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:48.274814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601bf5a-51f4-45bc-a3e1-f4f044c0eb31 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:48.274997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601bf5a-51f4-45bc-a3e1-f4f044c0eb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:48.292454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3601bf5a-51f4-45bc-a3e1-f4f044c0eb31 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:48.294153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3601bf5a-51f4-45bc-a3e1-f4f044c0eb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:48.301910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 740/1483] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:48.337817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34e4c-a3f8-4bf7-a3cf-418259520811 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:48.348443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc34e4c-a3f8-4bf7-a3cf-418259520811 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:48.351948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aaa2e-0d80-4894-b9b6-d81ba83a51fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:48.352602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aaa2e-0d80-4894-b9b6-d81ba83a51fb service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:48.352885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aaa2e-0d80-4894-b9b6-d81ba83a51fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:48.353160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aaa2e-0d80-4894-b9b6-d81ba83a51fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:48.362482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aaa2e-0d80-4894-b9b6-d81ba83a51fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:48.363929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aaa2e-0d80-4894-b9b6-d81ba83a51fb service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:48.373865 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 744/1484] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:50.640020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca1c59-cdd4-4b08-9d7c-209c0b066356 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:50.640575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca1c59-cdd4-4b08-9d7c-209c0b066356 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:50.640575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca1c59-cdd4-4b08-9d7c-209c0b066356 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:50.641911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 741/1485] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:50.647336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f688d8-0186-4b8e-a674-fff1d2e3467d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:50.649678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f688d8-0186-4b8e-a674-fff1d2e3467d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:50.649678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f688d8-0186-4b8e-a674-fff1d2e3467d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:50.666804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-93f688d8-0186-4b8e-a674-fff1d2e3467d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:50.668019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-93f688d8-0186-4b8e-a674-fff1d2e3467d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:50.676236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 745/1486] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:50.712995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca1c59-cdd4-4b08-9d7c-209c0b066356 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:50.723903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca1c59-cdd4-4b08-9d7c-209c0b066356 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:50.727576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95178350-f62f-4401-8a4a-bd772e0120dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:50.728390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178350-f62f-4401-8a4a-bd772e0120dc service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:50.728526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178350-f62f-4401-8a4a-bd772e0120dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:50.728816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95178350-f62f-4401-8a4a-bd772e0120dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:50.738327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178350-f62f-4401-8a4a-bd772e0120dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:50.739771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95178350-f62f-4401-8a4a-bd772e0120dc service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:50.749270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 742/1487] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:53.128991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7717b2b-2655-4d83-a2fe-0b5d06f14ec7 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:53.129564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7717b2b-2655-4d83-a2fe-0b5d06f14ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:53.129564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7717b2b-2655-4d83-a2fe-0b5d06f14ec7 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:53.130956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 746/1488] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:53.136324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23ef1-8247-4d8e-8001-20a79da847aa None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:53.136750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23ef1-8247-4d8e-8001-20a79da847aa None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:53.136895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23ef1-8247-4d8e-8001-20a79da847aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:53.154879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-83e23ef1-8247-4d8e-8001-20a79da847aa None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:53.156111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-83e23ef1-8247-4d8e-8001-20a79da847aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:53.163648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 743/1489] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:36:53.203617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7717b2b-2655-4d83-a2fe-0b5d06f14ec7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:53.214191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7717b2b-2655-4d83-a2fe-0b5d06f14ec7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:53.218194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f55356-fafc-4a04-b0f2-f8410530d5ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:53.218819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f55356-fafc-4a04-b0f2-f8410530d5ea service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:53.219106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f55356-fafc-4a04-b0f2-f8410530d5ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:53.219385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f55356-fafc-4a04-b0f2-f8410530d5ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:53.228080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f55356-fafc-4a04-b0f2-f8410530d5ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:53.229507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f55356-fafc-4a04-b0f2-f8410530d5ea service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:53.239752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 747/1490] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:36:55.890882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43480676-57fe-4442-af6f-960669abe7a2 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:55.891481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43480676-57fe-4442-af6f-960669abe7a2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:55.891481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43480676-57fe-4442-af6f-960669abe7a2 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:55.892848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 744/1491] 10.222.0.41 () {54 vars in 926 bytes} [Mon Jun 6 20:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:36:55.898102 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64191-d8cd-4acf-8631-7088bccf2760 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:55.898576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64191-d8cd-4acf-8631-7088bccf2760 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:55.898701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd64191-d8cd-4acf-8631-7088bccf2760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:55.916614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0dd64191-d8cd-4acf-8631-7088bccf2760 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:36:55.917796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0dd64191-d8cd-4acf-8631-7088bccf2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:36:55.926016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 748/1492] 10.222.0.41 () {58 vars in 1023 bytes} [Mon Jun 6 20:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:36:55.963346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43480676-57fe-4442-af6f-960669abe7a2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:36:55.974440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43480676-57fe-4442-af6f-960669abe7a2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:36:55.977956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c436b-eb7e-4ca2-968d-c9c08440d1d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:36:55.978583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c436b-eb7e-4ca2-968d-c9c08440d1d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:36:55.978871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c436b-eb7e-4ca2-968d-c9c08440d1d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:36:55.979151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c436b-eb7e-4ca2-968d-c9c08440d1d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:36:55.988456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c436b-eb7e-4ca2-968d-c9c08440d1d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:36:55.989879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47c436b-eb7e-4ca2-968d-c9c08440d1d5 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:36:55.999111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 745/1493] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:37:05.911565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=33 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:37:05.994956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=33 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:38:28.406340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.406904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.406904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.424279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:38:28.425446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:28.433546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 749/1494] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:38:28.441639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.442133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.442236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.460482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:38:28.461696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:28.469470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 746/1495] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:38:28.473885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:28.483297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d021c-2432-45ab-ac4c-0c8e1a2ac6b2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:28.487825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:28.488468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.488754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.489048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.489755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:38:28.490059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:28.491068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:28.491068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:38:28.491068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:38:28.491985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:28.503770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 750/1496] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:38:28.512203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:28.521245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa47f8-7607-4fff-948c-cd4be140a21e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:28.526038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:28.526699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.526987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.527268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.528000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:28.529739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:28.529739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:38:28.529739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:38:28.530927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:28.637372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 747/1497] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:38:28 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:38:28.641668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:28.651910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64c8b37-0699-4e67-9503-16f0dae83da8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:28.705682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:28.706259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.706544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.706819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.707502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:28.708531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:28.708531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:38:28.708531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:38:28.709717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:28.773166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 751/1498] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:38:28 2022] POST /identity/v3/users => generated 289 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:38:28.777237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:28.787199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a7609-17a0-416e-868b-956c00ddc72c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:28.791638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:28.792814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.793099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.793368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.794078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:28.795103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:28.795103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:38:28.795103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:38:28.796012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:28.808842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 748/1499] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:38:28.812678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:28.822375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d6b429-8d1a-44a8-be96-7743cde52d27 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:28.826844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:28.827489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.827814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.828095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.828813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:28.829817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:28.829817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:38:28.829817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:38:28.830699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418e43e-6a1f-47f7-ba90-a08e97b0bbbc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:28.844542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 752/1500] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:38:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:38:28.848526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:28.858684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567010e-1b51-481d-91b7-4cd9043cb7a4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:28.863111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:28.863997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.864277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.864565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] PATH_INFO: `/v3/projects/8e707add618f4f2c9fefea014c319b22/users/02aa8f6b5f4d4528835820682344dbc1/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.879237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e707add618f4f2c9fefea014c319b22, user_id=02aa8f6b5f4d4528835820682344dbc1, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:28.880282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:28.880282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:38:28.880282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:38:28.882198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60f38d-35f5-4bd6-a7f2-2d63483e5c31 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:28.944505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 749/1501] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:38:28 2022] PUT /identity/v3/projects/8e707add618f4f2c9fefea014c319b22/users/02aa8f6b5f4d4528835820682344dbc1/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:38:28.957066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31520c91-e2c8-4df7-87fa-2b080d8288b4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:28.957321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31520c91-e2c8-4df7-87fa-2b080d8288b4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:28.957612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31520c91-e2c8-4df7-87fa-2b080d8288b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:28.983743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-31520c91-e2c8-4df7-87fa-2b080d8288b4 None None] MFA Rules not processed for user `02aa8f6b5f4d4528835820682344dbc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:38:28.984932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-31520c91-e2c8-4df7-87fa-2b080d8288b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:29.045875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 753/1502] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:38:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:38:29.058692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af1b97-9f50-48d4-962e-2ade5ed3e40f None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.058963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af1b97-9f50-48d4-962e-2ade5ed3e40f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.059238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af1b97-9f50-48d4-962e-2ade5ed3e40f None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.060643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 750/1503] 10.222.0.41 () {54 vars in 963 bytes} [Mon Jun 6 20:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:38:29.065374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ea65f-147e-4b5e-9d1a-996ee3add240 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.065662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ea65f-147e-4b5e-9d1a-996ee3add240 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.065938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33ea65f-147e-4b5e-9d1a-996ee3add240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.082933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b33ea65f-147e-4b5e-9d1a-996ee3add240 None None] MFA Rules not processed for user `9110e7fab9084faa94b5d345bb9d519a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:38:29.084135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b33ea65f-147e-4b5e-9d1a-996ee3add240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:29.126243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:38:29.133369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 754/1504] 10.222.0.41 () {58 vars in 1060 bytes} [Mon Jun 6 20:38:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:38:29.140791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af1b97-9f50-48d4-962e-2ade5ed3e40f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:29.152041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af1b97-9f50-48d4-962e-2ade5ed3e40f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:29.155538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:29.156233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.156527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.156804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.165401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:29.167450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:29.176074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 751/1505] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:38:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:38:29.218129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f95fa9-2187-4c23-bfd8-ad4319f5e03d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.218412 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f95fa9-2187-4c23-bfd8-ad4319f5e03d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.218686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f95fa9-2187-4c23-bfd8-ad4319f5e03d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.220016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 755/1506] 10.222.0.41 () {52 vars in 829 bytes} [Mon Jun 6 20:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:38:29.223477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:29.286578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212809a3-73d6-4a75-a87a-e3eae211253d service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:29.291079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:29.291976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.292259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.292529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] PATH_INFO: `/v3/users/02aa8f6b5f4d4528835820682344dbc1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.293691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02aa8f6b5f4d4528835820682344dbc1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:29.294697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:29.294697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:38:29.294697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:38:29.295928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:29.482251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02aa8f6b5f4d4528835820682344dbc1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:29.482900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02aa8f6b5f4d4528835820682344dbc1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:29.544403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02aa8f6b5f4d4528835820682344dbc1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:29.556498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02aa8f6b5f4d4528835820682344dbc1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:29.572833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:38:29.575831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02aa8f6b5f4d4528835820682344dbc1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:29.582063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 752/1507] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:38:29 2022] DELETE /identity/v3/users/02aa8f6b5f4d4528835820682344dbc1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:38:29.586484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.588347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.588347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.604717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:38:29.605898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:29.662378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 756/1508] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:38:29.671678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:29.673120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:29.728830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25640f-efe2-4a1d-a792-b3840ffc8f88 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:29.732469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0c2c5-1778-4f98-8a47-7963d94967ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:29.733056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c2c5-1778-4f98-8a47-7963d94967ad service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:29.733343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c2c5-1778-4f98-8a47-7963d94967ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:29.733625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0c2c5-1778-4f98-8a47-7963d94967ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:29.742475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c2c5-1778-4f98-8a47-7963d94967ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:29.743941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0c2c5-1778-4f98-8a47-7963d94967ad service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:29.753785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 753/1509] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:38:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:38:30.209549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:38:30.211020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:38:30.222600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c1a2d-37e6-4dce-8fef-7c78561286d4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:38:30.227415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:38:30.228338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:38:30.228623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:38:30.228905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] PATH_INFO: `/v3/projects/8e707add618f4f2c9fefea014c319b22` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:38:30.232504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e707add618f4f2c9fefea014c319b22)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:38:30.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:38:30.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:38:30.233596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:38:30.234802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:38:30.353920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e707add618f4f2c9fefea014c319b22', 'request_id': 'req-e175c388-cf9e-4e40-8194-d31fcc27f3d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:30.424292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e707add618f4f2c9fefea014c319b22', 'request_id': 'req-e175c388-cf9e-4e40-8194-d31fcc27f3d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:30.430699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e707add618f4f2c9fefea014c319b22', 'request_id': 'req-e175c388-cf9e-4e40-8194-d31fcc27f3d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:30.448587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] The token cache is being invalidate because project 8e707add618f4f2c9fefea014c319b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:38:30.448870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:30.449497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e175c388-cf9e-4e40-8194-d31fcc27f3d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:38:30.451104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 757/1510] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:38:30 2022] DELETE /identity/v3/projects/8e707add618f4f2c9fefea014c319b22 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:38:39.749635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:38:40.445359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:40:05.936209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174a1c2-1ab8-4381-a7e2-b5ae2f5e8e29 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:05.936824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174a1c2-1ab8-4381-a7e2-b5ae2f5e8e29 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:05.936824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174a1c2-1ab8-4381-a7e2-b5ae2f5e8e29 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:05.938119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:05.942860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34689518-9da0-40c7-aed0-a2c42436a03b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:05.943272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34689518-9da0-40c7-aed0-a2c42436a03b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:05.943413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34689518-9da0-40c7-aed0-a2c42436a03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:05.961717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-34689518-9da0-40c7-aed0-a2c42436a03b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:05.962993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-34689518-9da0-40c7-aed0-a2c42436a03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:06.022409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:40:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:06.193376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174a1c2-1ab8-4381-a7e2-b5ae2f5e8e29 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:06.195044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b174a1c2-1ab8-4381-a7e2-b5ae2f5e8e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:06.272804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174a1c2-1ab8-4381-a7e2-b5ae2f5e8e29 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:06.278155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c090ac-c3aa-4d4b-a6b6-18444f9097e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:06.278803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c090ac-c3aa-4d4b-a6b6-18444f9097e4 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:06.279087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c090ac-c3aa-4d4b-a6b6-18444f9097e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:06.279367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c090ac-c3aa-4d4b-a6b6-18444f9097e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:06.287335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c090ac-c3aa-4d4b-a6b6-18444f9097e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:06.288967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c090ac-c3aa-4d4b-a6b6-18444f9097e4 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:06.298732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 755/1513] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:06.961165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c708c-9a91-463f-b274-aed7894dfee2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:06.961694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c708c-9a91-463f-b274-aed7894dfee2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:06.961795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c708c-9a91-463f-b274-aed7894dfee2 None None] PATH_INFO: `/v3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:06.963633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 759/1514] 10.222.0.32 () {54 vars in 925 bytes} [Mon Jun 6 20:40:06 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:40:06.969998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc7a5b-804e-4b00-be74-f9f50fd8ea15 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:06.970465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc7a5b-804e-4b00-be74-f9f50fd8ea15 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:06.970579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc7a5b-804e-4b00-be74-f9f50fd8ea15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:06.997441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-53fc7a5b-804e-4b00-be74-f9f50fd8ea15 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:06.997823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-53fc7a5b-804e-4b00-be74-f9f50fd8ea15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:07.068730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 756/1515] 10.222.0.32 () {58 vars in 1015 bytes} [Mon Jun 6 20:40:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:07.168411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03d4b-b671-45cd-963a-2b2f7ad7a48f None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:07.168705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03d4b-b671-45cd-963a-2b2f7ad7a48f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:07.168981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d03d4b-b671-45cd-963a-2b2f7ad7a48f None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:07.170735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:07.175337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a36d-0d6d-4136-9ef1-7626a57b0cc4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:07.175640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a36d-0d6d-4136-9ef1-7626a57b0cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:07.175958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72a36d-0d6d-4136-9ef1-7626a57b0cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:07.194427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-db72a36d-0d6d-4136-9ef1-7626a57b0cc4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:07.195631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-db72a36d-0d6d-4136-9ef1-7626a57b0cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:07.204330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:40:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:07.354996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d03d4b-b671-45cd-963a-2b2f7ad7a48f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:07.369881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d03d4b-b671-45cd-963a-2b2f7ad7a48f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:07.375348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e0624-7702-4ca7-a799-c6c2e8330fc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:07.376009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0624-7702-4ca7-a799-c6c2e8330fc7 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:07.376294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0624-7702-4ca7-a799-c6c2e8330fc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:07.376570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e0624-7702-4ca7-a799-c6c2e8330fc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:07.385264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0624-7702-4ca7-a799-c6c2e8330fc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:07.386827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e0624-7702-4ca7-a799-c6c2e8330fc7 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:07.396469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 761/1518] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:11.141411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db88e-347f-4bf6-8824-db203a56aa76 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:11.142015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db88e-347f-4bf6-8824-db203a56aa76 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:11.142015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2db88e-347f-4bf6-8824-db203a56aa76 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:11.143347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 758/1519] 10.222.0.41 () {54 vars in 960 bytes} [Mon Jun 6 20:40:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:11.148842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3885-0bf4-4142-b512-a4b8ec2da1a2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:11.149259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3885-0bf4-4142-b512-a4b8ec2da1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:11.149378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3885-0bf4-4142-b512-a4b8ec2da1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:11.170956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-aa5f3885-0bf4-4142-b512-a4b8ec2da1a2 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:11.172254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-aa5f3885-0bf4-4142-b512-a4b8ec2da1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:11.180900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 762/1520] 10.222.0.41 () {58 vars in 1057 bytes} [Mon Jun 6 20:40:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:40:11.188959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db88e-347f-4bf6-8824-db203a56aa76 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:11.202857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2db88e-347f-4bf6-8824-db203a56aa76 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:11.207776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e709c51e-f088-43bc-88f1-e5731aa31785 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:11.208412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709c51e-f088-43bc-88f1-e5731aa31785 service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:11.208702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709c51e-f088-43bc-88f1-e5731aa31785 service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:11.208984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e709c51e-f088-43bc-88f1-e5731aa31785 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:11.217412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709c51e-f088-43bc-88f1-e5731aa31785 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:11.218995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e709c51e-f088-43bc-88f1-e5731aa31785 service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:11.229545 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 759/1521] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:18.644026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f67e55-30e0-4caf-8fc8-c612e47c1d61 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:18.644543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f67e55-30e0-4caf-8fc8-c612e47c1d61 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:18.644543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f67e55-30e0-4caf-8fc8-c612e47c1d61 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:18.646208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:18.651514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128a1f8-687f-4acd-8884-8facf10b41fc None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:18.651927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128a1f8-687f-4acd-8884-8facf10b41fc None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:18.652094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6128a1f8-687f-4acd-8884-8facf10b41fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:18.670141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6128a1f8-687f-4acd-8884-8facf10b41fc None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:18.671313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6128a1f8-687f-4acd-8884-8facf10b41fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:18.678925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:40:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:18.829567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f67e55-30e0-4caf-8fc8-c612e47c1d61 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:18.844609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f67e55-30e0-4caf-8fc8-c612e47c1d61 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:18.849995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa17dc-381b-42cd-b81d-74d3c35da653 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:18.850638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa17dc-381b-42cd-b81d-74d3c35da653 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:18.850907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa17dc-381b-42cd-b81d-74d3c35da653 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:18.851181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa17dc-381b-42cd-b81d-74d3c35da653 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:18.860076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa17dc-381b-42cd-b81d-74d3c35da653 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:18.861647 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aa17dc-381b-42cd-b81d-74d3c35da653 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:18.871874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 764/1524] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:19.736056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838011e-c89a-49ef-b1f3-33b0055513d5 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:19.736713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838011e-c89a-49ef-b1f3-33b0055513d5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:19.736713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838011e-c89a-49ef-b1f3-33b0055513d5 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:19.738005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 6 20:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:19.742785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff800c1-7822-4cc0-b3d6-a735e1623f2e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:19.743230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff800c1-7822-4cc0-b3d6-a735e1623f2e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:19.743340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff800c1-7822-4cc0-b3d6-a735e1623f2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:19.761700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1ff800c1-7822-4cc0-b3d6-a735e1623f2e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:19.762967 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1ff800c1-7822-4cc0-b3d6-a735e1623f2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:19.771208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 6 20:40:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:19.919863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838011e-c89a-49ef-b1f3-33b0055513d5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:19.934496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838011e-c89a-49ef-b1f3-33b0055513d5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:19.939943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214efdb7-5566-4e8b-a169-6ea7b671af5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:19.940572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214efdb7-5566-4e8b-a169-6ea7b671af5e service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:19.940860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214efdb7-5566-4e8b-a169-6ea7b671af5e service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:19.941139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214efdb7-5566-4e8b-a169-6ea7b671af5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:19.949841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214efdb7-5566-4e8b-a169-6ea7b671af5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:19.951417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214efdb7-5566-4e8b-a169-6ea7b671af5e service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:19.960749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 762/1527] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:23.842417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60c19-1d31-4d85-ad7d-2dec58ea5f08 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:23.842965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60c19-1d31-4d85-ad7d-2dec58ea5f08 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:23.843082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f60c19-1d31-4d85-ad7d-2dec58ea5f08 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:23.844503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:23.849948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f8bf9-cbf4-483a-8b48-f61b7df00a87 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:23.850415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f8bf9-cbf4-483a-8b48-f61b7df00a87 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:23.850531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f8bf9-cbf4-483a-8b48-f61b7df00a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:23.868960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-e20f8bf9-cbf4-483a-8b48-f61b7df00a87 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:23.870158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e20f8bf9-cbf4-483a-8b48-f61b7df00a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:23.877852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:40:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:23.916511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f60c19-1d31-4d85-ad7d-2dec58ea5f08 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:23.918294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-92f60c19-1d31-4d85-ad7d-2dec58ea5f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:23.976380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f60c19-1d31-4d85-ad7d-2dec58ea5f08 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:23.980589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7a054-9438-4e05-93aa-d2824cc06895 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:23.982329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7a054-9438-4e05-93aa-d2824cc06895 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:23.982329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7a054-9438-4e05-93aa-d2824cc06895 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:23.982329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7a054-9438-4e05-93aa-d2824cc06895 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:23.990714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7a054-9438-4e05-93aa-d2824cc06895 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:23.992178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7a054-9438-4e05-93aa-d2824cc06895 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:24.001555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 767/1530] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:26.220429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865cb9-d69f-499c-8fb2-a727ee39e9be None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:26.221063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865cb9-d69f-499c-8fb2-a727ee39e9be None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:26.221063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b865cb9-d69f-499c-8fb2-a727ee39e9be None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:26.222391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:40:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:26.228379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cab3b-a237-436b-a9ac-d9c9a8263429 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:26.229172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cab3b-a237-436b-a9ac-d9c9a8263429 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:26.229459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95cab3b-a237-436b-a9ac-d9c9a8263429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:26.248074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c95cab3b-a237-436b-a9ac-d9c9a8263429 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:26.249355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c95cab3b-a237-436b-a9ac-d9c9a8263429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:26.257247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:26.292803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865cb9-d69f-499c-8fb2-a727ee39e9be None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:26.303281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b865cb9-d69f-499c-8fb2-a727ee39e9be None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:26.307403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba06ea-2618-4117-9e74-663336d390c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:26.308052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba06ea-2618-4117-9e74-663336d390c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:26.308333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba06ea-2618-4117-9e74-663336d390c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:26.308603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba06ea-2618-4117-9e74-663336d390c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:26.317248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba06ea-2618-4117-9e74-663336d390c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:26.318661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba06ea-2618-4117-9e74-663336d390c1 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:26.327922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 765/1533] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:40:30.156106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b9e6-9d1d-4dcb-80f0-cb349a103d3d None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:30.156661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b9e6-9d1d-4dcb-80f0-cb349a103d3d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:30.156661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352b9e6-9d1d-4dcb-80f0-cb349a103d3d None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:30.158067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:40:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:30.163339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35cac4-c09c-4227-87f4-e134bc8ecabf None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:30.163851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35cac4-c09c-4227-87f4-e134bc8ecabf None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:30.163953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c35cac4-c09c-4227-87f4-e134bc8ecabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:30.170831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=91 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:40:30.183097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0c35cac4-c09c-4227-87f4-e134bc8ecabf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:30.190714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0c35cac4-c09c-4227-87f4-e134bc8ecabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:30.192422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:40:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:30.459097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352b9e6-9d1d-4dcb-80f0-cb349a103d3d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:30.460836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9352b9e6-9d1d-4dcb-80f0-cb349a103d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:30.518349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352b9e6-9d1d-4dcb-80f0-cb349a103d3d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:30.522163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a42659-5007-46cd-aa6a-71a5cf5bcafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:30.522793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a42659-5007-46cd-aa6a-71a5cf5bcafd service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:30.523078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a42659-5007-46cd-aa6a-71a5cf5bcafd service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:30.523354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a42659-5007-46cd-aa6a-71a5cf5bcafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:30.532236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a42659-5007-46cd-aa6a-71a5cf5bcafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:30.533647 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a42659-5007-46cd-aa6a-71a5cf5bcafd service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:30.538571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:40:30.543165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 770/1536] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:32.791842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a20-8f6c-4f66-9712-f13dc778381c None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:32.792393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a20-8f6c-4f66-9712-f13dc778381c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:32.792393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab0a20-8f6c-4f66-9712-f13dc778381c None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:32.793782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 6 20:40:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:40:32.799002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133c47a-658c-4446-8121-04cadfa22f0a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:32.799425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133c47a-658c-4446-8121-04cadfa22f0a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:32.799586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133c47a-658c-4446-8121-04cadfa22f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:32.817324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b133c47a-658c-4446-8121-04cadfa22f0a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:40:32.818584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b133c47a-658c-4446-8121-04cadfa22f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:32.826352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 6 20:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:40:33.093729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0a20-8f6c-4f66-9712-f13dc778381c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:40:33.095450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-edab0a20-8f6c-4f66-9712-f13dc778381c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:40:33.108202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab0a20-8f6c-4f66-9712-f13dc778381c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:40:33.111639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9b8dc-07ab-40a3-a189-2eab636595ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:40:33.112288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9b8dc-07ab-40a3-a189-2eab636595ea service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:40:33.112593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9b8dc-07ab-40a3-a189-2eab636595ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:40:33.112856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9b8dc-07ab-40a3-a189-2eab636595ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:40:33.121531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9b8dc-07ab-40a3-a189-2eab636595ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:40:33.122936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9b8dc-07ab-40a3-a189-2eab636595ea service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:40:33.132165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 768/1539] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:40:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:40:42.812113 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:40:43.128150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=8 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:02.022087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877b486-7192-46d1-9c7a-d5459f21b12b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.023581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877b486-7192-46d1-9c7a-d5459f21b12b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.024121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877b486-7192-46d1-9c7a-d5459f21b12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.035017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe3e15-32dd-4aac-90ae-4b556b8c3991 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.035651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe3e15-32dd-4aac-90ae-4b556b8c3991 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.035651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe3e15-32dd-4aac-90ae-4b556b8c3991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.075173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d877b486-7192-46d1-9c7a-d5459f21b12b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.076983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d877b486-7192-46d1-9c7a-d5459f21b12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.078003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-74fe3e15-32dd-4aac-90ae-4b556b8c3991 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.079704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-74fe3e15-32dd-4aac-90ae-4b556b8c3991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.094892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 772/1541] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.095089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 769/1541] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.103487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.103725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.104157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.104406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.104833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.104993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.142189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.143407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.145026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.146347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.156901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 770/1542] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.162358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.167687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 773/1543] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.173578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.177691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19522e6-af16-4ea9-b16c-7bd9dba510b9 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.183111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.183809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.184098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.184377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.185104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.185410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.186430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:42:02.186430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:42:02.186430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:42:02.187316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b8f730-a417-4bba-a325-b96828022831 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.189661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4395ef00-64a2-4627-bd00-75af29fe134f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.201658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.202872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.203163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.203436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.204197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.204501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.206580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.211129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 771/1544] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:42:02.212458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.212751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.213034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.225242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 774/1545] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:42:02.234235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.255555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73864155-415e-4a44-9dc6-036dc471afec admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.263037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.264408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.272374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.273080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.273370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.273630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 772/1546] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.273939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.274320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.274829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.275907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:42:02.275907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:42:02.275907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:42:02.277115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9092d3c-811d-4cb0-be66-8f4e14c80eb9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.290299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68aed4-3978-4b2a-8e63-68c0c9d6a9f5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.297133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb6424-993e-4a06-8b67-a2afc6356077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.297805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb6424-993e-4a06-8b67-a2afc6356077 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.298090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb6424-993e-4a06-8b67-a2afc6356077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.298366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb6424-993e-4a06-8b67-a2afc6356077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.301888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb6424-993e-4a06-8b67-a2afc6356077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.304217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb6424-993e-4a06-8b67-a2afc6356077 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.379334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 775/1547] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:02.381242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42400ed0-256c-47eb-8088-6cf1ea1b8cdf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.381852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42400ed0-256c-47eb-8088-6cf1ea1b8cdf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.382437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42400ed0-256c-47eb-8088-6cf1ea1b8cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.405087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 773/1548] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/projects => generated 371 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:02.406627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb9da-c6f6-4d74-b934-48fa44f83ba2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.407145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb9da-c6f6-4d74-b934-48fa44f83ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.407396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-42400ed0-256c-47eb-8088-6cf1ea1b8cdf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.407704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aeb9da-c6f6-4d74-b934-48fa44f83ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.409159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-42400ed0-256c-47eb-8088-6cf1ea1b8cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.436675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-43aeb9da-c6f6-4d74-b934-48fa44f83ba2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.438792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-43aeb9da-c6f6-4d74-b934-48fa44f83ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.474860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 776/1549] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.476079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.476369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.476650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.498426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.499757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.503478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 774/1550] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.504753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e77351-4dc3-4dcf-924b-50e09401e394 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.505045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e77351-4dc3-4dcf-924b-50e09401e394 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.505324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e77351-4dc3-4dcf-924b-50e09401e394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.507711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 777/1551] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.508692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.518628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfad594-dea5-4687-8d48-d2e7e19c27cd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.523733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dff7a2-2650-40a3-8cce-79d99cb92144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.524319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff7a2-2650-40a3-8cce-79d99cb92144 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.524596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff7a2-2650-40a3-8cce-79d99cb92144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.524864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dff7a2-2650-40a3-8cce-79d99cb92144 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.525539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dff7a2-2650-40a3-8cce-79d99cb92144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.528015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dff7a2-2650-40a3-8cce-79d99cb92144 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.531373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-e5e77351-4dc3-4dcf-924b-50e09401e394 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.532604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e5e77351-4dc3-4dcf-924b-50e09401e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.540465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 775/1552] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.542263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.542550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.542828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.560610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.561696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.569411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 776/1553] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.570134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.579357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3a991f-f7e2-4049-b1a7-e2cd856e18df None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.583893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.584522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.584808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.585083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.585751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.586776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:42:02.586776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:42:02.586776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:42:02.588005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca3b26-620e-4ff5-9db9-e5c681b9ee62 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.603195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 778/1554] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:02.605063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.605349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.605624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.622981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.624222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.631907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 779/1555] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.632662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.641532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a5cdc8-82e2-43c2-b91c-61800caa3668 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.646191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.646837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.647119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.647413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.648127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.648428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.650006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ffc33-3c72-46ad-8469-8668bb9abe0f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.661714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 780/1556] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:02.662857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684beb0-55da-4374-9049-8e4c82d0cc94 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.663141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684beb0-55da-4374-9049-8e4c82d0cc94 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.663416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684beb0-55da-4374-9049-8e4c82d0cc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.680911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8684beb0-55da-4374-9049-8e4c82d0cc94 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.682229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8684beb0-55da-4374-9049-8e4c82d0cc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.683081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 777/1557] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:02.684262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482b0de-78b1-45e1-9691-28156d158ba1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.684589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482b0de-78b1-45e1-9691-28156d158ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.687009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5482b0de-78b1-45e1-9691-28156d158ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.690133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 781/1558] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.691334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.691620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.691930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.705948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5482b0de-78b1-45e1-9691-28156d158ba1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.707224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5482b0de-78b1-45e1-9691-28156d158ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.708904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.710254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.715866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 778/1559] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.717015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9c604-813c-4087-8b5c-de17c5c796c5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.717303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9c604-813c-4087-8b5c-de17c5c796c5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.717576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc9c604-813c-4087-8b5c-de17c5c796c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.720742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 782/1560] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.721505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.731786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8278ff27-d081-4cc2-8382-22bc067ae299 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.736326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.737003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.737286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.737615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.738367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.739972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.740475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fdc9c604-813c-4087-8b5c-de17c5c796c5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.741655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fdc9c604-813c-4087-8b5c-de17c5c796c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.749670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 779/1561] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.750381 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 783/1562] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:02.750958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.751250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.751527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.753792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.763371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456dfe89-f5d6-4ad3-8c77-40e74d67f3d7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.767880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.768503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.769062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.769224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:02.769345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.769990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.770394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:02.771877 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.779170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 780/1563] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:02.780418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.789848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc94301b-127c-477b-9ad0-f6656674e59c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.794943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.800493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.800493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.800493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.800493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.800493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.808177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 781/1564] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:02.809052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.818341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfcb7d0-9247-4b04-b5db-2462702f15f3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.822797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.823377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.823677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.823957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.824634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.824937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.826487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.838090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 782/1565] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:02.839389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.849008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403c2d-00b3-4cdb-b5af-ad2bad1b65a5 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.853550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.854757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.855048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.855348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.855897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 784/1566] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:02.856106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.856381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.856830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.858017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.876822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 783/1567] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:02.876822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.880179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86739140-3036-431f-9f00-bd8878cb51c4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.890626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:02.893536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:02.927224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b33c2-e2aa-4c6d-ac94-b17d0ebb30e6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.932241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.932933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.933216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.933491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.934206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.934515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.936128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.937315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.938021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.938353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.938631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.939373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.940975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.948687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 785/1568] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:02.949565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.952329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 784/1569] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:02.952329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.959252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b8ac8-24f4-4fc2-aa34-9c12282c4fd4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.961764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00084-d9f3-491a-81c6-5e2b043b5e20 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.964600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.965239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.965523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.965797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.966638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.966799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.966926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.967441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.967742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.968015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.968553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.968712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:02.969028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:02.970594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:02.981016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 786/1570] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:02.999008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.999156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 785/1571] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:02.999156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:02.999343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd604d-f05a-46d3-ab79-0455768dec0d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.999445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794f781-6506-49ca-8c03-24e827071a57 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:02.999549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:02.999549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:02.999549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:02.999549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:02.999922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.000024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.000122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.000122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.000122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.000457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.000562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.002339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.011490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 787/1572] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:03.012654 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.023061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0db54a-dbf0-4c47-b0f2-425577084a26 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.028356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.029005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.029339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.029567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.030245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.032258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.100832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 786/1573] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:02 2022] POST /identity/v3/projects => generated 379 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.101637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.128072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 788/1574] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.130557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.138438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee62dd-5920-4d0c-b6d2-416990d7989e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.167157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35508d-c6df-455d-9475-ac89ffb836e6 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.171730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.172389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.172810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.173101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.173789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.175710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.195580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.196275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.196553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.196824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.197550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.200007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.270812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 787/1575] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/projects => generated 377 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.270812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.278261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 789/1576] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/projects => generated 387 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.279058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.332987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e1e42-f9ca-43b1-b36b-f34dfe7e833a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.337525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.338435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.338717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.339002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] PATH_INFO: `/v3/projects/09d0b71cb0264e56a2193698af985783/users/cd06ee464a92494fb955e8c7e8bd8ff2/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.341481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795a9a8-1ece-4b0c-8b95-1294e2859dd9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.346137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.346782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.347070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.347217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:03.347338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.348044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.349972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.353294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:03.358486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d0b71cb0264e56a2193698af985783, user_id=cd06ee464a92494fb955e8c7e8bd8ff2, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.358486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.465070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 788/1577] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:03 2022] PUT /identity/v3/projects/09d0b71cb0264e56a2193698af985783/users/cd06ee464a92494fb955e8c7e8bd8ff2/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:03.465866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.480048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 790/1578] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/projects => generated 383 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.480854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.528458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318aa84-e30c-484f-846e-ff7f2e230585 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.540067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e005a-e0cc-4eb7-bf8d-8cd18f8651b8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.544963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.545827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.546104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.546373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] PATH_INFO: `/v3/projects/79a53829d5ad4d8ba2ffda13b9a1b376/users/4db9c6cc7635402cb3c325d659c2ba9a/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.562261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a53829d5ad4d8ba2ffda13b9a1b376, user_id=4db9c6cc7635402cb3c325d659c2ba9a, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.563394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:42:03.563394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:42:03.563394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:42:03.565441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.584680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.585273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.585554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.585832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.586507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.588421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.665394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 791/1579] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:03 2022] PUT /identity/v3/projects/79a53829d5ad4d8ba2ffda13b9a1b376/users/4db9c6cc7635402cb3c325d659c2ba9a/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:03.666398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.701356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 789/1580] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/projects => generated 391 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.703895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.728957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55162273-540e-49b0-8dca-6d9eca2bde7f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.763347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa12a58-2c7c-4013-83f2-b7a5999bcd39 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.768183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.769135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.769415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.769685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.770401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.772066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.781809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 790/1581] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:03.782632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.789439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.790295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.790578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.790853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.791537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.791849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14061f-18d3-4d95-b25d-01e207b97a1b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.794055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.796794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa42a61-0fca-4286-bd56-23ed0fbe11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.797413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa42a61-0fca-4286-bd56-23ed0fbe11d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.797695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa42a61-0fca-4286-bd56-23ed0fbe11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.797968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa42a61-0fca-4286-bd56-23ed0fbe11d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.798631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa42a61-0fca-4286-bd56-23ed0fbe11d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.800535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa42a61-0fca-4286-bd56-23ed0fbe11d7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.911136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 792/1582] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/users => generated 290 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.911985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:03.917663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:03.918471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:03.923565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec5d27-5e82-46da-a97d-c401f3d6e59b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:03.926804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 791/1583] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:03.926804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7fd73-bf72-4bf1-b904-7247306238dd None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.926804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7fd73-bf72-4bf1-b904-7247306238dd None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.926804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d7fd73-bf72-4bf1-b904-7247306238dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.928531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:03.929283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:03.929571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:03.929843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:03.930556 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:03.932584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:03.951384 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-87d7fd73-bf72-4bf1-b904-7247306238dd None None] MFA Rules not processed for user `cd06ee464a92494fb955e8c7e8bd8ff2`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:03.952608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-87d7fd73-bf72-4bf1-b904-7247306238dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:04.014509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 793/1584] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/users => generated 294 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.014950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 792/1585] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:04.016240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.016529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.016806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.019286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.029032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7886a0eb-8285-4c6c-ad10-7752f969a9ef admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.033545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.034191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.034473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.034744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.035753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.037651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.042651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] MFA Rules not processed for user `4db9c6cc7635402cb3c325d659c2ba9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:04.043904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:04.104687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 793/1586] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:04.105529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.115344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd41c0-3ba0-4d3b-b291-17d0302b0e3d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.120328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.121629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.122011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.122287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.122962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 794/1587] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.123082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.123848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.125016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.134444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08564300-9981-4cb1-a6bc-e6ed8afd5472 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.139933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.140585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.140870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.141144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.141898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.143479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.154260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 795/1588] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.155051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.164208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae6f2f4-c93e-4ca5-b025-136c73a30ed4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.168653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.169259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.169542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.169817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.170504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.172067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.181589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 796/1589] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.183760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.194141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f82146-d2d2-4a2c-aa36-237ed2d72e23 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.198622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.199907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.200194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.200470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.201246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.202796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.209492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 794/1590] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.209492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.213309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 797/1591] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.213984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.222747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efbe7f-d6dd-4ea4-9a06-8852e851ed1e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.227792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4bc98c-2255-498f-b169-f43adbd050b3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.227949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.228512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.228797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.228912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.229069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.229574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.230660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.230897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.231256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.231821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.232125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.234746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.245431 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 795/1592] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.246747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.256106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdea96c-4010-4567-8912-38b658f41a37 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.260710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.262990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.262990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.262990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.263308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.265071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.317974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 798/1593] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.318939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.362511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 796/1594] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.362511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.380181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646a3266-1178-4caa-b3ee-05016875d1cc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.389730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:04.421439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9d57-c8ac-4f44-89c4-1879bc7d7566 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.427024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.427943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.428233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.428531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] PATH_INFO: `/v3/projects/eb56f6104d014f75bc5809840c2277a6/users/772261cd4bbe4e07a20919e144f9291b/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.436902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.439430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.439430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.439430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.439430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.440525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.443230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb56f6104d014f75bc5809840c2277a6, user_id=772261cd4bbe4e07a20919e144f9291b, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.445885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.450308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 799/1595] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.451109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.460224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32e7a3-5c3e-4942-8ccf-8749e5464075 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.464907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.465564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.465842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.466115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.466845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.468510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.478033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 800/1596] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.478986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.488669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f74e0b-8dfa-4488-8311-74226eee7e2f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.493520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.497507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.497507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.497507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.497507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.497507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.498725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:04.500723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 797/1597] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:04 2022] PUT /identity/v3/projects/eb56f6104d014f75bc5809840c2277a6/users/772261cd4bbe4e07a20919e144f9291b/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:04.501539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.511061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 801/1598] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.511061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.521180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c1d34-89e5-4e19-a50e-0d89a9a0275e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.571830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10daf3d-d032-45c2-8575-db41ae11ba55 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.576557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.577271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.577510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.577647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.577861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.578162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.578432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.578619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.578723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.579421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.580260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.581492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.590043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 798/1599] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.590880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.591700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 802/1600] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.595368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.600664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fa447-9676-4c06-897b-67ccf9262309 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.604559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea209b-d942-488d-92d1-0efdf7e28b4b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.605756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.606393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.606678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.606952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.607616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.609532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.609783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.610415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.610698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.610976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.611637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.613533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.682331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 799/1601] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.683928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.693602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23379cec-cd06-4809-8526-78d87051a80d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.694530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 803/1602] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:04.695353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.698397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.699143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.699431 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.699746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.700471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.702070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.704622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16767c0-77b0-478e-83ad-fd1772df6c5c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.709161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.710020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.714320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.714320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] PATH_INFO: `/v3/projects/7baa5449b82446198b16e944162aa85b/users/9583c43bffb24329b5e06b1e749affd8/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.714560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 800/1603] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.714560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.725476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7baa5449b82446198b16e944162aa85b, user_id=9583c43bffb24329b5e06b1e749affd8, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.725644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb8e8d-97fd-4f84-8b9a-b4d7ebd1c338 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.728201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.730650 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef39f74c-e1ba-4e50-92dc-7b26a10ff6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.731293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39f74c-e1ba-4e50-92dc-7b26a10ff6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.731578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39f74c-e1ba-4e50-92dc-7b26a10ff6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.731894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef39f74c-e1ba-4e50-92dc-7b26a10ff6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.732605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39f74c-e1ba-4e50-92dc-7b26a10ff6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.734172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef39f74c-e1ba-4e50-92dc-7b26a10ff6ef admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.743960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 801/1604] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:04.745126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.745419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.746248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.770908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] MFA Rules not processed for user `772261cd4bbe4e07a20919e144f9291b`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:04.772137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:04.802832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 804/1605] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:04 2022] PUT /identity/v3/projects/7baa5449b82446198b16e944162aa85b/users/9583c43bffb24329b5e06b1e749affd8/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:04.803902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.829894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:04.838677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 802/1606] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:04.840170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:04.850419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e75a9e1-bbb3-45b0-aeb2-3c83203c32ae None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.873042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc543b-6480-42bb-8d11-a7e8880179ef admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:04.877721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.878605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.878889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.879166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] PATH_INFO: `/v3/projects/3a1af880b45d426aa752589183354a77/users/a47ab3ebbc9a4a429e5df2a616f305f0/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.894445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a1af880b45d426aa752589183354a77, user_id=a47ab3ebbc9a4a429e5df2a616f305f0, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.897156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:04.910088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:04.910968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:04.911251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:04.911524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] PATH_INFO: `/v3/projects/1e2a93c20b2e4ace88184d8b156a0575/users/3d3896bd70dd43fe95f7125f9f4693a0/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:04.925779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e2a93c20b2e4ace88184d8b156a0575, user_id=3d3896bd70dd43fe95f7125f9f4693a0, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:04.928456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.002395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 805/1607] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:04 2022] PUT /identity/v3/projects/3a1af880b45d426aa752589183354a77/users/a47ab3ebbc9a4a429e5df2a616f305f0/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.003332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.026236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 803/1608] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:04 2022] PUT /identity/v3/projects/1e2a93c20b2e4ace88184d8b156a0575/users/3d3896bd70dd43fe95f7125f9f4693a0/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.028221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.039636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:05.066661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d17dd6-e91f-42e9-84e5-b63a94f6d87f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.088057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac46327-7d7c-4137-b74f-2ad2f17d0eba admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.093151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.093814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.094104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.094382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.095098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.096732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.107325 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 804/1609] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:05.108703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.119271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def3d6ea-0cea-43db-bac6-8c60076cd5ec admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.122814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.124038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.124038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.124220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68cdad-eb44-4596-a85a-ae19a97fee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.124353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.124849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.124978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68cdad-eb44-4596-a85a-ae19a97fee5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.125257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68cdad-eb44-4596-a85a-ae19a97fee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.125538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68cdad-eb44-4596-a85a-ae19a97fee5f admin admin] PATH_INFO: `/v3/projects/f3894355fc424d08abe9e2fe0f66a7a7/users/f80b80a88c2d4e57a58eaafd431d4e93/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.126511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.139797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 806/1610] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:05.139797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.141732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68cdad-eb44-4596-a85a-ae19a97fee5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3894355fc424d08abe9e2fe0f66a7a7, user_id=f80b80a88c2d4e57a58eaafd431d4e93, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.144736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68cdad-eb44-4596-a85a-ae19a97fee5f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.149525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719e83c-1a72-4dad-8f2b-188852e52cc0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.154704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece3f5f-9ed1-4369-9401-4176714169ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.155599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece3f5f-9ed1-4369-9401-4176714169ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.155922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece3f5f-9ed1-4369-9401-4176714169ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.156194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece3f5f-9ed1-4369-9401-4176714169ff admin admin] PATH_INFO: `/v3/projects/ffb14a29d13a483ba8733e0bb3a56a17/users/a10ffb60631240fdb6b4a69d6d7f73e2/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.170853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece3f5f-9ed1-4369-9401-4176714169ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb14a29d13a483ba8733e0bb3a56a17, user_id=a10ffb60631240fdb6b4a69d6d7f73e2, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.173767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece3f5f-9ed1-4369-9401-4176714169ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.208394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 805/1611] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:05 2022] PUT /identity/v3/projects/f3894355fc424d08abe9e2fe0f66a7a7/users/f80b80a88c2d4e57a58eaafd431d4e93/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.209723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad03932-d4dd-44ec-9a96-cf3c0b1e5e74 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.210045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad03932-d4dd-44ec-9a96-cf3c0b1e5e74 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.210319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad03932-d4dd-44ec-9a96-cf3c0b1e5e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.230836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6ad03932-d4dd-44ec-9a96-cf3c0b1e5e74 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.232403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6ad03932-d4dd-44ec-9a96-cf3c0b1e5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.249604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 807/1612] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:05 2022] PUT /identity/v3/projects/ffb14a29d13a483ba8733e0bb3a56a17/users/a10ffb60631240fdb6b4a69d6d7f73e2/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.250899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bac17-8ca1-4a9b-9d69-7c62988d3489 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.251242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bac17-8ca1-4a9b-9d69-7c62988d3489 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.251525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7bac17-8ca1-4a9b-9d69-7c62988d3489 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.255746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 808/1613] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:05.258073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.258363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.258644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.276237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.278077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.280573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:05.294165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 806/1614] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.295402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.295715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.295994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.316125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.317326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.338388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 809/1615] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.340077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.349903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c183110-6d6a-4399-9e35-c9af5b9f0a8a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.354636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcad43d-25dc-46ca-8bd4-3070ab249c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.357984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcad43d-25dc-46ca-8bd4-3070ab249c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.357984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcad43d-25dc-46ca-8bd4-3070ab249c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.357984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcad43d-25dc-46ca-8bd4-3070ab249c52 admin admin] PATH_INFO: `/v3/projects/5ccd3a696fd645bba0d3d21a3140beda/users/ae446cdf6c1a4bba8c76cf08ecdef954/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.370722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcad43d-25dc-46ca-8bd4-3070ab249c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccd3a696fd645bba0d3d21a3140beda, user_id=ae446cdf6c1a4bba8c76cf08ecdef954, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.373489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcad43d-25dc-46ca-8bd4-3070ab249c52 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.378372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 807/1616] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.379333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.389211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87096c-028b-497f-bc3c-0f199d3a3d34 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.394188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd34ed-723c-4130-857e-126ce0a3e83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.395050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd34ed-723c-4130-857e-126ce0a3e83d admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.395377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd34ed-723c-4130-857e-126ce0a3e83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.395653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd34ed-723c-4130-857e-126ce0a3e83d admin admin] PATH_INFO: `/v3/projects/eb681e8653924d51a25f6e1ca4d6fff1/users/8ff78c0bfc024c7a944af3725d376905/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.410908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd34ed-723c-4130-857e-126ce0a3e83d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb681e8653924d51a25f6e1ca4d6fff1, user_id=8ff78c0bfc024c7a944af3725d376905, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.413609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd34ed-723c-4130-857e-126ce0a3e83d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.442705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 810/1617] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:05 2022] PUT /identity/v3/projects/5ccd3a696fd645bba0d3d21a3140beda/users/ae446cdf6c1a4bba8c76cf08ecdef954/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.444026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7288ba-2798-49a3-9a5a-edb8d0a22b5f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.444316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7288ba-2798-49a3-9a5a-edb8d0a22b5f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.444592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7288ba-2798-49a3-9a5a-edb8d0a22b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.469503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9c7288ba-2798-49a3-9a5a-edb8d0a22b5f None None] MFA Rules not processed for user `f80b80a88c2d4e57a58eaafd431d4e93`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.470784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c7288ba-2798-49a3-9a5a-edb8d0a22b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.471269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 808/1618] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:05 2022] PUT /identity/v3/projects/eb681e8653924d51a25f6e1ca4d6fff1/users/8ff78c0bfc024c7a944af3725d376905/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.474948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.474948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.474948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.488220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:05.491140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.492615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.534920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 811/1619] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.536462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.536753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.537032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.553527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 809/1620] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.554599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.565089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.566888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.610985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaf567-202e-4085-99e8-86b13e417b53 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.614771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.615446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.615767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.616049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.625596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.627082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.634708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 812/1621] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.636597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.637594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 810/1622] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:05.638394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.646354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4de45-f299-4d93-a115-b2e48286f3cb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.647161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac93760f-a53e-49b8-b82a-ca3ae5796a3d service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.650343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.650761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.651006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.651269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.651407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.651586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.651717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.652506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.662914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.663573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.664387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.665190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.669067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:05.675531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 811/1623] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:05.676121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 813/1624] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:05.676348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.676846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.687389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c2d8c-8a2b-43cd-9c8e-0b7dc9aea513 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.687538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4acd9f-9f10-4e51-8125-e56001d35058 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.691291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f64a01-63b0-447f-9733-d789a8f26387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.692008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f64a01-63b0-447f-9733-d789a8f26387 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.692279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.692434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f64a01-63b0-447f-9733-d789a8f26387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.692585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f64a01-63b0-447f-9733-d789a8f26387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.693298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.693413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f64a01-63b0-447f-9733-d789a8f26387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.693712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.694062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.694857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.694969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f64a01-63b0-447f-9733-d789a8f26387 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.696606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.705477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 812/1625] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:05.706639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c9845-ee44-4db5-b5a3-e35a9be67ba3 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.707281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c9845-ee44-4db5-b5a3-e35a9be67ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.707281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8c9845-ee44-4db5-b5a3-e35a9be67ba3 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.708732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 813/1626] 10.222.0.41 () {54 vars in 961 bytes} [Mon Jun 6 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:05.709455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c9845-ee44-4db5-b5a3-e35a9be67ba3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.711352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 814/1627] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:05.712239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.723733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8c9845-ee44-4db5-b5a3-e35a9be67ba3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.727104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3e3b-3672-44b9-8c55-39fd347609b2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.727977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.728600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.728926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.729161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.734309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.735016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.735375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.735757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.739633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.741126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.750608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 814/1628] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:05.751371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.760309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2192530-da76-4de1-bf99-b8475d26017e service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.765135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a944324-9c30-4651-a43e-e9dcc25f9cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.765912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944324-9c30-4651-a43e-e9dcc25f9cae admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.766196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944324-9c30-4651-a43e-e9dcc25f9cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.766472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a944324-9c30-4651-a43e-e9dcc25f9cae admin admin] PATH_INFO: `/v3/projects/5ccd3a696fd645bba0d3d21a3140beda/users/ae446cdf6c1a4bba8c76cf08ecdef954/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.769202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944324-9c30-4651-a43e-e9dcc25f9cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ccd3a696fd645bba0d3d21a3140beda, user_id=ae446cdf6c1a4bba8c76cf08ecdef954, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.771886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a944324-9c30-4651-a43e-e9dcc25f9cae admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.784730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 815/1629] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:05 2022] PUT /identity/v3/projects/5ccd3a696fd645bba0d3d21a3140beda/users/ae446cdf6c1a4bba8c76cf08ecdef954/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.786211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0194504-1c19-4694-a18b-2baf00dfc1aa None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.786447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0194504-1c19-4694-a18b-2baf00dfc1aa None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.786727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0194504-1c19-4694-a18b-2baf00dfc1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.807870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.809587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.810813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-e0194504-1c19-4694-a18b-2baf00dfc1aa None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.812145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e0194504-1c19-4694-a18b-2baf00dfc1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.824823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 815/1630] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:05.828792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:05.846142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc751fc-ee86-4952-b8ba-bd4cdf4336f8 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:05.877415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:05.906042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 816/1631] 10.222.0.41 () {58 vars in 1058 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:05.908367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a08d-dd1b-489b-bea4-e96845765725 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.908673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a08d-dd1b-489b-bea4-e96845765725 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.908960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9a08d-dd1b-489b-bea4-e96845765725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.932308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:05.933615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:05.934005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:05.934396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] PATH_INFO: `/v3/projects/eb681e8653924d51a25f6e1ca4d6fff1/users/8ff78c0bfc024c7a944af3725d376905/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:05.937969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb681e8653924d51a25f6e1ca4d6fff1, user_id=8ff78c0bfc024c7a944af3725d376905, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:05.941344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:05.947644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d3f9a08d-dd1b-489b-bea4-e96845765725 None None] MFA Rules not processed for user `ae446cdf6c1a4bba8c76cf08ecdef954`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:05.951026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d3f9a08d-dd1b-489b-bea4-e96845765725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:05.966155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 816/1632] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:05 2022] PUT /identity/v3/projects/eb681e8653924d51a25f6e1ca4d6fff1/users/8ff78c0bfc024c7a944af3725d376905/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:05.966155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:06.033504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 817/1633] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:06.035140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.035583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.036028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.068864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a691f-df98-4678-9db8-eea555e38dbe admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:06.075819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] MFA Rules not processed for user `8ff78c0bfc024c7a944af3725d376905`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:06.077035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:06.078304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f053d283-5dc0-4f0d-b81d-3d8855ecb93e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:06.079185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053d283-5dc0-4f0d-b81d-3d8855ecb93e service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.079573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053d283-5dc0-4f0d-b81d-3d8855ecb93e service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.079986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f053d283-5dc0-4f0d-b81d-3d8855ecb93e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.107522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:06.148705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053d283-5dc0-4f0d-b81d-3d8855ecb93e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:06.151023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f053d283-5dc0-4f0d-b81d-3d8855ecb93e service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:06.160617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 818/1634] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:06.161748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:06.174818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 817/1635] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:06.174818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184445fd-3c7a-4f72-b7c6-863b2a4c93b2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.174818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184445fd-3c7a-4f72-b7c6-863b2a4c93b2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.174818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184445fd-3c7a-4f72-b7c6-863b2a4c93b2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.174818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 818/1636] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 20:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:06.174818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184445fd-3c7a-4f72-b7c6-863b2a4c93b2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:06.262802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184445fd-3c7a-4f72-b7c6-863b2a4c93b2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:06.268947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd0746-215c-49e0-8722-19399b760ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:06.269662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0746-215c-49e0-8722-19399b760ebb service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.270228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0746-215c-49e0-8722-19399b760ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.271413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd0746-215c-49e0-8722-19399b760ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.281133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83331f02-97d9-4521-8b2a-3616161bbab5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:06.286156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0746-215c-49e0-8722-19399b760ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:06.286712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a10988-c4f2-4899-93dd-d07a0873d928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:06.287888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10988-c4f2-4899-93dd-d07a0873d928 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.288368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd0746-215c-49e0-8722-19399b760ebb service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:06.288847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10988-c4f2-4899-93dd-d07a0873d928 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.289296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a10988-c4f2-4899-93dd-d07a0873d928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.293485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:06.302971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a10988-c4f2-4899-93dd-d07a0873d928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:06.309008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a10988-c4f2-4899-93dd-d07a0873d928 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:06.312284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 819/1637] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:06.313960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f3d6-fc3f-458d-b9d9-f7adcf9dfd82 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.314246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f3d6-fc3f-458d-b9d9-f7adcf9dfd82 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.314521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986f3d6-fc3f-458d-b9d9-f7adcf9dfd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.333595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 819/1638] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:06.348672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0986f3d6-fc3f-458d-b9d9-f7adcf9dfd82 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:06.350112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0986f3d6-fc3f-458d-b9d9-f7adcf9dfd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:06.371247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 820/1639] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:42:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:06.415086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f30ae-6297-423b-9fff-4d8c3a5e69a4 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.415378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f30ae-6297-423b-9fff-4d8c3a5e69a4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.415679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f30ae-6297-423b-9fff-4d8c3a5e69a4 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.417075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 820/1640] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 20:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:06.422280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.422695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.423805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.441772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:06.443682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:06.459993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f30ae-6297-423b-9fff-4d8c3a5e69a4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:06.461777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 821/1641] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:42:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:06.471986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f30ae-6297-423b-9fff-4d8c3a5e69a4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:06.475839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:06.481120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.481120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.481120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.488617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:06.490032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:06.501842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 821/1642] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:06.665099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:06.677596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d6d56-48c6-4f8f-8b8b-34fa5618031c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:06.682426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233490e7-2d63-4c45-b65e-fc6a82378b2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:06.683995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233490e7-2d63-4c45-b65e-fc6a82378b2f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:06.684494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233490e7-2d63-4c45-b65e-fc6a82378b2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:06.685237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233490e7-2d63-4c45-b65e-fc6a82378b2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:06.697834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233490e7-2d63-4c45-b65e-fc6a82378b2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:06.699749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233490e7-2d63-4c45-b65e-fc6a82378b2f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:06.711831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 822/1643] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:07.680181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:07.689709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd474a2a-7091-4245-81c7-fa50dad54c83 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:07.696638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c531a10-7b09-482a-8208-cce67a2ddd9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:07.697300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531a10-7b09-482a-8208-cce67a2ddd9b service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:07.697583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531a10-7b09-482a-8208-cce67a2ddd9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:07.697865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c531a10-7b09-482a-8208-cce67a2ddd9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:07.855215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c531a10-7b09-482a-8208-cce67a2ddd9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:07.855215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c531a10-7b09-482a-8208-cce67a2ddd9b service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:07.876997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 822/1644] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:07.895321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:07.895880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:07.895880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:07.986060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] MFA Rules not processed for user `9583c43bffb24329b5e06b1e749affd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:07.988336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:08.092206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 823/1645] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:08.094215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:08.094215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:08.094547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:08.123849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] MFA Rules not processed for user `9583c43bffb24329b5e06b1e749affd8`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:08.125207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:08.139711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 823/1646] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:08.172206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:08.204955 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908c4b97-884e-4cd8-8b77-4723f92710d8 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:08.210764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc8045-a932-4da1-91d3-4828b72e2ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:08.211435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8045-a932-4da1-91d3-4828b72e2ec2 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:08.211744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8045-a932-4da1-91d3-4828b72e2ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:08.212043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc8045-a932-4da1-91d3-4828b72e2ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:08.244591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc8045-a932-4da1-91d3-4828b72e2ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:08.244591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc8045-a932-4da1-91d3-4828b72e2ec2 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:08.267519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 824/1647] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:08 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:09.127588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:09.201593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:09.216649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85967-14d3-44ee-a4fc-6182ebb490dd None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:09.217502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85967-14d3-44ee-a4fc-6182ebb490dd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:09.217948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f85967-14d3-44ee-a4fc-6182ebb490dd None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:09.219884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 825/1648] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 20:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:09.224817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:09.225375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:09.225827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:09.250077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c9895-2a73-48f8-a91f-c1c404b6a7e8 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:09.266636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:09.267844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:09.268264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:09.268965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] PATH_INFO: `/v3/users/4db9c6cc7635402cb3c325d659c2ba9a` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:09.270329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db9c6cc7635402cb3c325d659c2ba9a)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:09.272928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:09.329643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:09.330859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:09.439755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 826/1649] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:09.471722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:09.483527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aff564-aed8-4706-bb96-9e1ee1e915e1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:09.490701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f7fec-d0b0-4b1f-8cea-00b691db1337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:09.492827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7fec-d0b0-4b1f-8cea-00b691db1337 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:09.493175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7fec-d0b0-4b1f-8cea-00b691db1337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:09.493445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f7fec-d0b0-4b1f-8cea-00b691db1337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:09.511697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7fec-d0b0-4b1f-8cea-00b691db1337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:09.513611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f7fec-d0b0-4b1f-8cea-00b691db1337 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:09.523836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 827/1650] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:09.620084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db9c6cc7635402cb3c325d659c2ba9a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:09.620882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db9c6cc7635402cb3c325d659c2ba9a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:09.723117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db9c6cc7635402cb3c325d659c2ba9a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:09.731990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db9c6cc7635402cb3c325d659c2ba9a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:09.744967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db9c6cc7635402cb3c325d659c2ba9a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:09.756249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 824/1651] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:09 2022] DELETE /identity/v3/users/4db9c6cc7635402cb3c325d659c2ba9a => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:09.756971 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a34f9-e75d-45b9-81c6-1413c2d98ab7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:09.757250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a34f9-e75d-45b9-81c6-1413c2d98ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:09.757523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a34f9-e75d-45b9-81c6-1413c2d98ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:09.785692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6d7a34f9-e75d-45b9-81c6-1413c2d98ab7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:09.787497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6d7a34f9-e75d-45b9-81c6-1413c2d98ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:09.865051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:09.920240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 828/1652] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:09.933497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:09.934960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:10.117551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53171dcd-111d-42de-beea-5a473a3b8540 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:10.123537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13cacca-4b56-46cf-80a7-ab666de61f82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:10.124285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cacca-4b56-46cf-80a7-ab666de61f82 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:10.124575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cacca-4b56-46cf-80a7-ab666de61f82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:10.124920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13cacca-4b56-46cf-80a7-ab666de61f82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:10.196588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cacca-4b56-46cf-80a7-ab666de61f82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:10.204424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13cacca-4b56-46cf-80a7-ab666de61f82 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:10.220662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 825/1653] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:10.369357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae95ab-f67a-447a-bcc6-6c8019375ca7 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:10.369357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae95ab-f67a-447a-bcc6-6c8019375ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:10.369357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae95ab-f67a-447a-bcc6-6c8019375ca7 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:10.369357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 829/1654] 10.222.0.41 () {54 vars in 960 bytes} [Mon Jun 6 20:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:10.372558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:10.372820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:10.373105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:10.394583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:10.395956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:10.506538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 826/1655] 10.222.0.41 () {58 vars in 1057 bytes} [Mon Jun 6 20:42:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:10.507142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae95ab-f67a-447a-bcc6-6c8019375ca7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:10.507730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae95ab-f67a-447a-bcc6-6c8019375ca7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:10.523871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:10.524537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:10.524822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:10.525098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:10.527096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:10.545962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:10.551224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:10.583726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 830/1656] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:11.822581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:11.824575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:11.837625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:11.849681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29d628-162b-4bb5-bfc5-7650abf8580d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:11.860650 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:11.863399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:11.866680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:11.869739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] PATH_INFO: `/v3/projects/79a53829d5ad4d8ba2ffda13b9a1b376` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:11.876959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a53829d5ad4d8ba2ffda13b9a1b376)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:11.882831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: WARNING py.warnings [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:42:11.882831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: warnings.warn(msg) Jun 06 20:42:11.882831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: Jun 06 20:42:11.885000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:12.025133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a53829d5ad4d8ba2ffda13b9a1b376', 'request_id': 'req-d52272a0-059e-4c1c-9a39-a7927869f7dd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.100044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a53829d5ad4d8ba2ffda13b9a1b376', 'request_id': 'req-d52272a0-059e-4c1c-9a39-a7927869f7dd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.114452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a53829d5ad4d8ba2ffda13b9a1b376', 'request_id': 'req-d52272a0-059e-4c1c-9a39-a7927869f7dd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.157913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] The token cache is being invalidate because project 79a53829d5ad4d8ba2ffda13b9a1b376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:12.158467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.159811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.177542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:12.180284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 827/1657] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:11 2022] DELETE /identity/v3/projects/79a53829d5ad4d8ba2ffda13b9a1b376 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:12.197916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66063da2-350e-4605-bcaa-d940866927d1 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:12.349544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:12.350884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:12.351171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:12.351450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] PATH_INFO: `/v3/users/ae446cdf6c1a4bba8c76cf08ecdef954` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:12.360772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae446cdf6c1a4bba8c76cf08ecdef954)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:12.361836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: WARNING py.warnings [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 06 20:42:12.361836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: warnings.warn(msg) Jun 06 20:42:12.361836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: Jun 06 20:42:12.368563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:12.729699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae446cdf6c1a4bba8c76cf08ecdef954'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.731365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae446cdf6c1a4bba8c76cf08ecdef954'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.866026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae446cdf6c1a4bba8c76cf08ecdef954'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.908814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae446cdf6c1a4bba8c76cf08ecdef954'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.927799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2d038df-c432-45e9-935c-8279a618b750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae446cdf6c1a4bba8c76cf08ecdef954'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:12.940471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 831/1658] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:12 2022] DELETE /identity/v3/users/ae446cdf6c1a4bba8c76cf08ecdef954 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:13.718721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:13.721052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:13.755431 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52272a0-059e-4c1c-9a39-a7927869f7dd admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:13.883751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:13.883751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:13.883751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:13.883751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] PATH_INFO: `/v3/users/cd06ee464a92494fb955e8c7e8bd8ff2` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:13.885162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd06ee464a92494fb955e8c7e8bd8ff2)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:13.887894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:14.090519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd06ee464a92494fb955e8c7e8bd8ff2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.091099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd06ee464a92494fb955e8c7e8bd8ff2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.145642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd06ee464a92494fb955e8c7e8bd8ff2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.158492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd06ee464a92494fb955e8c7e8bd8ff2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.195945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd06ee464a92494fb955e8c7e8bd8ff2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.214205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 828/1659] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:13 2022] DELETE /identity/v3/users/cd06ee464a92494fb955e8c7e8bd8ff2 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:14.218576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:14.219059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:14.219341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:14.292321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:14.300187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:14.343737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:14.431854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 832/1660] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:14.508357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:14.510085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:14.520990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:14.527939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:14.539185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:14.546860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5cc8-fa21-493e-a971-a59a89a38852 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:14.574802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:14.581132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:14.583334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:14.587853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] PATH_INFO: `/v3/projects/5ccd3a696fd645bba0d3d21a3140beda` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:14.594353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ccd3a696fd645bba0d3d21a3140beda)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:14.607259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:14.657647 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d025f64b-2e26-455c-9049-69a5bc96cebf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:14.678931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:14.679608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:14.679948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:14.680233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:14.702972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:14.704710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:14.723378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 833/1661] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:14.724243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:14.727484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:14.745901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d408730d-dffa-40bc-b022-bbfb48bcc00e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:14.751406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:14.752974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:14.753266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:14.753663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] PATH_INFO: `/v3/users/772261cd4bbe4e07a20919e144f9291b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:14.756617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] RBAC: Authorizing `identity:delete_user(user_id=772261cd4bbe4e07a20919e144f9291b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:14.757756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:14.799948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ccd3a696fd645bba0d3d21a3140beda', 'request_id': 'req-300b8de2-4fb3-4184-9599-70e907315dad'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.876831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ccd3a696fd645bba0d3d21a3140beda', 'request_id': 'req-300b8de2-4fb3-4184-9599-70e907315dad'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.885789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ccd3a696fd645bba0d3d21a3140beda', 'request_id': 'req-300b8de2-4fb3-4184-9599-70e907315dad'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.911130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] The token cache is being invalidate because project 5ccd3a696fd645bba0d3d21a3140beda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:14.911463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.912171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-300b8de2-4fb3-4184-9599-70e907315dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:14.919324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 829/1662] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:14 2022] DELETE /identity/v3/projects/5ccd3a696fd645bba0d3d21a3140beda => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:14.921106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105d7e-b38f-468b-a164-87605fa79b17 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:14.921400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105d7e-b38f-468b-a164-87605fa79b17 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:14.921676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4105d7e-b38f-468b-a164-87605fa79b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:14.962213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c4105d7e-b38f-468b-a164-87605fa79b17 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:14.963387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c4105d7e-b38f-468b-a164-87605fa79b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:15.055973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '772261cd4bbe4e07a20919e144f9291b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:15.056652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '772261cd4bbe4e07a20919e144f9291b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:15.103308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 830/1663] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:15.106899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.106899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.111176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.137700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:15.141187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:15.151157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '772261cd4bbe4e07a20919e144f9291b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:15.155032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 831/1664] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:15.156437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:15.174114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e49524-a84c-4954-80cc-5864318b6ebb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:15.184596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:15.186353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.186353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.187157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.188460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '772261cd4bbe4e07a20919e144f9291b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:15.188577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:15.189033 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:15.192119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:15.213254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-92f5facc-9164-406c-a0f1-ff03ef95382e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '772261cd4bbe4e07a20919e144f9291b'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:15.218453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 832/1665] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:15.224719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:15.225215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 834/1666] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:14 2022] DELETE /identity/v3/users/772261cd4bbe4e07a20919e144f9291b => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:15.234360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.234644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.234933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.246506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f015aa-da3b-406e-b530-02e8332a42f4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:15.290313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:15.291718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:15.412120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:15.412120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.412120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.412429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.415048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:15.437099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:15.441777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:15.460311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 835/1667] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:15.557758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:15.563360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:15.566752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:15.597554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 833/1668] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:15 2022] POST /identity/v3/projects => generated 363 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:15.601936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:15.683970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c35a4-b5f4-4a3b-a8f2-c4052b127298 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:15.838994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced687c9-6452-4927-a37f-5e4ad7d8af89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:15.840448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced687c9-6452-4927-a37f-5e4ad7d8af89 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.840953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced687c9-6452-4927-a37f-5e4ad7d8af89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.841404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced687c9-6452-4927-a37f-5e4ad7d8af89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.856646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced687c9-6452-4927-a37f-5e4ad7d8af89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:15.863284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753419c-454d-40f8-b1f9-69bff19b3f7f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:15.864557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced687c9-6452-4927-a37f-5e4ad7d8af89 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:15.877521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:15.881490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.881875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.882455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.883442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:15.890373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:15.890640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 836/1669] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:15.892215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:15.892563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:15.892860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:15.928975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:15.933083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:15.997212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 834/1670] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:15 2022] POST /identity/v3/users => generated 282 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:16.001398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:16.033666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c4a62-5b0e-4237-bce1-101f82deeda8 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:16.059004 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:16.064990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:16.069709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:16.069709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:16.077487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:16.079040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:16.113882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 837/1671] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:42:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:16.126191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 835/1672] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:15 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:16.131126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:16.153142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38543650-7884-489d-a4fd-79edd4069a6d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:16.160136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b144aa0-b66e-4f01-9111-9710a0a0692f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:16.162730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144aa0-b66e-4f01-9111-9710a0a0692f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:16.163306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144aa0-b66e-4f01-9111-9710a0a0692f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:16.163902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b144aa0-b66e-4f01-9111-9710a0a0692f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:16.164928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144aa0-b66e-4f01-9111-9710a0a0692f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:16.168314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b144aa0-b66e-4f01-9111-9710a0a0692f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:16.188136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 838/1673] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:16.191372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:16.206597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1aa8a-6c27-458d-89af-411e44ae1c59 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:16.227817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:16.227817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:16.228082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:16.228295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] PATH_INFO: `/v3/projects/bbba9a6248554e9ab398f3b55c59ce68/users/b804c67483e34dcab9a0fedab8388b27/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:16.272363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbba9a6248554e9ab398f3b55c59ce68, user_id=b804c67483e34dcab9a0fedab8388b27, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:16.275132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:16.369499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 836/1674] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:16 2022] PUT /identity/v3/projects/bbba9a6248554e9ab398f3b55c59ce68/users/b804c67483e34dcab9a0fedab8388b27/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:16.381835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:16.382699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:16.383126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:16.440235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] MFA Rules not processed for user `b804c67483e34dcab9a0fedab8388b27`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:16.444303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:16.524211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:16.525913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:16.527304 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:16.618722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 839/1675] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:16 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:16.632816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:16.719033 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83055dc-c8b6-4236-8173-d6d493ef353d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:16.731564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:16.732725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:16.732725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:16.733096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:16.805660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177a7143-debc-4acd-808a-e7ae319ecc47 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:16.817208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:16.819601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:16.819969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:16.820316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:16.824822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:16.827847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:16.852135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:16.866482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:16.868282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:16.879339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 837/1676] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:42:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 20:42:16.941618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 840/1677] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:16 2022] POST /identity/v3/projects => generated 365 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:16.942100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:16.968081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a651119-a127-45bb-9e4a-32fc3ef03f7e service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:17.128666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:17.132529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:17.134860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:17.137277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:17.141465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:17.149707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:17.294715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 838/1678] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:16 2022] POST /identity/v3/users => generated 289 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:17.297165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:17.312428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3468cda-171a-4759-8020-eacbd6157797 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:17.321634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:17.322521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:17.322829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:17.323090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:17.324497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:17.326824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:17.346310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 841/1679] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:17.350582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:17.367972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82e6bab-9daf-43e7-8e97-3e9849f38fb3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:17.375729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:17.376969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:17.377334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:17.378180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] PATH_INFO: `/v3/projects/cc87dddd9a7440afb8d80ae9e97f9143/users/ee5f9d7f29da47658821ddea73df84da/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:17.424732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc87dddd9a7440afb8d80ae9e97f9143, user_id=ee5f9d7f29da47658821ddea73df84da, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:17.429202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:17.545569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 839/1680] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:17 2022] PUT /identity/v3/projects/cc87dddd9a7440afb8d80ae9e97f9143/users/ee5f9d7f29da47658821ddea73df84da/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:17.550390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:17.584254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc70e42-7cac-42e3-9694-5b8f0cf60555 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:17.799906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:17.803253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:17.803930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:17.804411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:17.805406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:17.812058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:17.858606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:17.865979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 842/1681] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:17.897667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b1a4c-16fd-4161-a380-4c65b330f32a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:17.934167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138eb54-1c7f-4b6e-a77f-795392534b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:17.935089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138eb54-1c7f-4b6e-a77f-795392534b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:17.938808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138eb54-1c7f-4b6e-a77f-795392534b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:17.938808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138eb54-1c7f-4b6e-a77f-795392534b2b admin admin] PATH_INFO: `/v3/projects/cc87dddd9a7440afb8d80ae9e97f9143/users/ee5f9d7f29da47658821ddea73df84da/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:17.965572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:17.970226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:17.972059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138eb54-1c7f-4b6e-a77f-795392534b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc87dddd9a7440afb8d80ae9e97f9143, user_id=ee5f9d7f29da47658821ddea73df84da, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:17.974873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138eb54-1c7f-4b6e-a77f-795392534b2b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:17.989686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0128d341-9961-40e9-aff0-730a9c795cf2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:17.995369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:17.999830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:18.000386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:18.000862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] PATH_INFO: `/v3/users/f80b80a88c2d4e57a58eaafd431d4e93` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:18.003430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f80b80a88c2d4e57a58eaafd431d4e93)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:18.008449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:18.074108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 840/1682] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:17 2022] PUT /identity/v3/projects/cc87dddd9a7440afb8d80ae9e97f9143/users/ee5f9d7f29da47658821ddea73df84da/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:18.076819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:18.076819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:18.077049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:18.210076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] MFA Rules not processed for user `ee5f9d7f29da47658821ddea73df84da`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:18.211518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:18.240494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:18.263963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f80b80a88c2d4e57a58eaafd431d4e93'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:18.266937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f80b80a88c2d4e57a58eaafd431d4e93'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:18.515822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 841/1683] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:18.535551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:18.538030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:18.600724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:18.649936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f80b80a88c2d4e57a58eaafd431d4e93'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:18.672191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea2b8e-2ddb-4d33-bbb7-6177b46d3485 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:18.686796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:18.686796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:18.686796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:18.686796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:18.695404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f80b80a88c2d4e57a58eaafd431d4e93'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:18.715811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:18.720143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:18.723952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6df72734-6bd1-4d34-8f61-a44f20fe872d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f80b80a88c2d4e57a58eaafd431d4e93'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:18.739714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 843/1684] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:17 2022] DELETE /identity/v3/users/f80b80a88c2d4e57a58eaafd431d4e93 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:18.744710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:18.744999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:18.745310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:18.781532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 842/1685] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:18.812913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:18.814182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:18.971862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 844/1686] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:18.990588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:18.992464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:19.198230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a3fce6-8ede-4936-b71d-f6566c8afd29 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:19.326494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:19.326494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:19.326494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:19.326494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:19.382530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:19.384028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:19.403720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 843/1687] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:19.931690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:19.933755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:19.956305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9beaf280-dd74-45fb-b0b8-4569ab0bb390 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:19.958102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:19.959535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:19.979811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a670eb-bab2-4584-b75c-b4d5be9d3aee service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:19.990098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:19.993277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:19.993847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:19.994690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] PATH_INFO: `/v3/projects/eb56f6104d014f75bc5809840c2277a6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:19.997475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:19.998360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:19.998658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:19.998927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] PATH_INFO: `/v3/projects/09d0b71cb0264e56a2193698af985783` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:20.001688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb56f6104d014f75bc5809840c2277a6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:20.011800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:20.015103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d0b71cb0264e56a2193698af985783)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:20.021698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:20.174890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb56f6104d014f75bc5809840c2277a6', 'request_id': 'req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.233021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d0b71cb0264e56a2193698af985783', 'request_id': 'req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.271015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb56f6104d014f75bc5809840c2277a6', 'request_id': 'req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.285823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb56f6104d014f75bc5809840c2277a6', 'request_id': 'req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.356670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] The token cache is being invalidate because project eb56f6104d014f75bc5809840c2277a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:20.357116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d0b71cb0264e56a2193698af985783', 'request_id': 'req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.357886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.361310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.368662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 845/1688] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:19 2022] DELETE /identity/v3/projects/eb56f6104d014f75bc5809840c2277a6 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:20.373317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:20.377988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:20.379858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d0b71cb0264e56a2193698af985783', 'request_id': 'req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.419321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] The token cache is being invalidate because project 09d0b71cb0264e56a2193698af985783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:20.419614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.428115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf99ab6b-d018-4fbf-bb24-366a6138e1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:20.429854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 844/1689] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:19 2022] DELETE /identity/v3/projects/09d0b71cb0264e56a2193698af985783 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:20.434752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:20.434752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:20.434752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:20.521849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:20.527577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:20.540703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:20.574624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:20.629871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc703846-5c7d-43ae-b440-9e5bcc9ed2b2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:20.684682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 845/1690] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:20.686800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:20.690413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:20.724303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6495619e-a57c-4a2b-b960-bb6448ca2f2d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:20.740200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:20.741388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:20.741680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:20.742745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] PATH_INFO: `/v3/users/8ff78c0bfc024c7a944af3725d376905` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:20.744840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ff78c0bfc024c7a944af3725d376905)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:20.749236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:20.804275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:20.810144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:20.810144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:20.810144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:20.815766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:20.969356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:20.971762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810fe187-fdcb-467b-bafc-afa9fef282f8 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:21.016278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 846/1691] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:20 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:21.018019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a58c8-3bcc-4fde-a50f-fc5fa1672d1b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:21.020686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a58c8-3bcc-4fde-a50f-fc5fa1672d1b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:21.020686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a58c8-3bcc-4fde-a50f-fc5fa1672d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:21.079291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-235a58c8-3bcc-4fde-a50f-fc5fa1672d1b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:21.090118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-235a58c8-3bcc-4fde-a50f-fc5fa1672d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:21.093207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ff78c0bfc024c7a944af3725d376905'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:21.095399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ff78c0bfc024c7a944af3725d376905'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:21.156253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 847/1692] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:21.156253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:21.156253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:21.156253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:21.244175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:21.246750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:21.453511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 848/1693] 10.222.0.32 () {58 vars in 1015 bytes} [Mon Jun 6 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:21.454337 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:21.488234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ff78c0bfc024c7a944af3725d376905'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:21.508252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea69ed11-83fb-4ad7-90ee-1a3ee5f7514e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:21.509879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ff78c0bfc024c7a944af3725d376905'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:21.520051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:21.522154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:21.523146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:21.523511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:21.535581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:21.536997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:21.539029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:21.567802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ff78c0bfc024c7a944af3725d376905'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:21.578106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 849/1694] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:21.579271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:21.590915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 846/1695] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:20 2022] DELETE /identity/v3/users/8ff78c0bfc024c7a944af3725d376905 => generated 0 bytes in 905 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:21.594432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:21.594635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:21.631749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4623b-fe0d-41cb-908b-42ea4c5cbbe6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:21.779890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e832edb-c5e3-401e-87d9-1174e84e9f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:21.780763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e832edb-c5e3-401e-87d9-1174e84e9f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:21.781150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e832edb-c5e3-401e-87d9-1174e84e9f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:21.781538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e832edb-c5e3-401e-87d9-1174e84e9f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:21.805415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e832edb-c5e3-401e-87d9-1174e84e9f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:21.811156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e832edb-c5e3-401e-87d9-1174e84e9f40 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:21.811313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d95cec-e82f-45a5-bae4-4b073bb8eec4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:21.844148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 850/1696] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:21.844834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:21.846368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:21.846907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:21.847701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:21.848377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:21.848658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:21.848933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:21.849630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:21.851545 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:21.913849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] MFA Rules not processed for user `3d3896bd70dd43fe95f7125f9f4693a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:21.916424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:21.981686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 847/1697] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:21 2022] POST /identity/v3/projects => generated 405 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:21.985696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:21.997577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:22.030263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:22.060194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 851/1698] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:22.068033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:22.077160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06457b5-86d6-40ef-95ca-ef9a9b30c2ef None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:22.162057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:22.162815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.163106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.163900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.165209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:22.168115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:22.208643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:22.234538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44401d88-754e-4ccb-ae2f-bcf39fe35116 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:22.262558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:22.268154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.268154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.268154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] PATH_INFO: `/v3/projects/f3894355fc424d08abe9e2fe0f66a7a7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.268154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3894355fc424d08abe9e2fe0f66a7a7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:22.283464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:22.300076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 852/1699] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:22 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:22.300076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:22.302585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:22.449224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdbbf6f-0fd3-4cf1-a3d0-971e72fe8d82 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:22.458089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:22.459964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.460432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.460863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.564449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3894355fc424d08abe9e2fe0f66a7a7', 'request_id': 'req-54676432-0d5f-4915-acd0-ec6cdbabbd7c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:22.587008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:22.589622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:22.606557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 853/1700] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:22.608694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:22.626151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac24d415-55c5-435f-bcc8-2a441ae92857 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:22.661839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3894355fc424d08abe9e2fe0f66a7a7', 'request_id': 'req-54676432-0d5f-4915-acd0-ec6cdbabbd7c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:22.671024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3894355fc424d08abe9e2fe0f66a7a7', 'request_id': 'req-54676432-0d5f-4915-acd0-ec6cdbabbd7c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:22.706075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] The token cache is being invalidate because project f3894355fc424d08abe9e2fe0f66a7a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:22.706387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:22.713044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-54676432-0d5f-4915-acd0-ec6cdbabbd7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:22.717198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db2e23-121b-4dd2-8427-b9bef995364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:22.717915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db2e23-121b-4dd2-8427-b9bef995364d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.718221 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db2e23-121b-4dd2-8427-b9bef995364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.718503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db2e23-121b-4dd2-8427-b9bef995364d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.719834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db2e23-121b-4dd2-8427-b9bef995364d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:22.721648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db2e23-121b-4dd2-8427-b9bef995364d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:22.724440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 848/1701] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:21 2022] DELETE /identity/v3/projects/f3894355fc424d08abe9e2fe0f66a7a7 => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:22.729337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.732438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.733569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.738430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 854/1702] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:22.738430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.738430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.738430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.788706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:22.796991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:22.822075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 855/1703] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:22.833954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] MFA Rules not processed for user `a10ffb60631240fdb6b4a69d6d7f73e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:22.838273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:22.839635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:22.844154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:22.887987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887c8ed7-68ba-4780-8cba-f967d8353179 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:22.894838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:22.907403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b3c9f-8aa9-4840-96a5-633b82716acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:22.909281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b3c9f-8aa9-4840-96a5-633b82716acb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.909743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b3c9f-8aa9-4840-96a5-633b82716acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.910206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b3c9f-8aa9-4840-96a5-633b82716acb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.911116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b3c9f-8aa9-4840-96a5-633b82716acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:22.914720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b3c9f-8aa9-4840-96a5-633b82716acb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:22.935911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 856/1704] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:22.937494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:22.937793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:22.938078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:22.993305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:22.995279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:23.042893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 857/1705] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:23.048523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:23.070366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 849/1706] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:23.072724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:23.073806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9696f5c4-4d6e-426a-b658-3e0e955563a6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:23.085404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ba219-6dd5-443a-b878-81b06f93b0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:23.089433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ba219-6dd5-443a-b878-81b06f93b0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.089724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ba219-6dd5-443a-b878-81b06f93b0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.090007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ba219-6dd5-443a-b878-81b06f93b0f8 admin admin] PATH_INFO: `/v3/projects/952b2a2d9302405882681f6ca82b0dde/users/66260219b02d418791a99cd1c9049b74/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:23.133793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ba219-6dd5-443a-b878-81b06f93b0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952b2a2d9302405882681f6ca82b0dde, user_id=66260219b02d418791a99cd1c9049b74, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:23.140078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63e5522-7a85-4413-93ba-d43eaa7d83e3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:23.141954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441ba219-6dd5-443a-b878-81b06f93b0f8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:23.174641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:23.174641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.174641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.174641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:23.174942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:23.174942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:23.187649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:23.233271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 858/1707] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:23 2022] PUT /identity/v3/projects/952b2a2d9302405882681f6ca82b0dde/users/66260219b02d418791a99cd1c9049b74/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:23.234027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 850/1708] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:23.234130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f3fa-ae29-40c9-8553-a7320192f6d4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.234130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f3fa-ae29-40c9-8553-a7320192f6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.234130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f3fa-ae29-40c9-8553-a7320192f6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:23.234793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:23.269916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-be12f3fa-ae29-40c9-8553-a7320192f6d4 None None] MFA Rules not processed for user `a10ffb60631240fdb6b4a69d6d7f73e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:23.271256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-be12f3fa-ae29-40c9-8553-a7320192f6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:23.282253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f2f8d-ff4c-47e7-8ce0-68921a3f3097 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:23.460328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 859/1709] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:23.460328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.466586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.467024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:23.512827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:23.512827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:23.513078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:23.513078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.517386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.517688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:23.521094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:23.529087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:23.536465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 860/1710] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:23.546610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:23.546610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:23.728741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 851/1711] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:23 2022] POST /identity/v3/projects => generated 381 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:23.734261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:23.735094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f901f3-74bc-44c8-a280-351c1c82c5ba None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:23.740623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:23.971227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4bdf38-18c2-4a37-92e5-d11019af4b05 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:23.986098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:23.986944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:23.987640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.988010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.988292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:23.991573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:23.996778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:23.997685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.277472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:24.280197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:24.293320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 861/1712] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 756 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:24.294640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:24.367751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:24.376305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:24.388999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea181412-ce1a-4679-8239-e64b85c221fd service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:24.396925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:24.397630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.397921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.418210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 852/1713] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:24.420953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:24.421377 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.422492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:24.428175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:24.428427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:24.477687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4a22e-21be-4e4c-a3ef-67e66e786e92 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:24.498167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:24.499823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.500331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.500818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] PATH_INFO: `/v3/projects/eb681e8653924d51a25f6e1ca4d6fff1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.509047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb681e8653924d51a25f6e1ca4d6fff1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:24.512703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:24.527421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 862/1714] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:24 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:24.529672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:24.552468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7cf27-a81f-4a8c-bb6a-6060c3e96451 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:24.571420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcbee92-5bcc-4f23-b414-59fa490b8e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:24.583874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbee92-5bcc-4f23-b414-59fa490b8e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.586014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbee92-5bcc-4f23-b414-59fa490b8e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.587804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcbee92-5bcc-4f23-b414-59fa490b8e4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.591910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbee92-5bcc-4f23-b414-59fa490b8e4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:24.609087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcbee92-5bcc-4f23-b414-59fa490b8e4d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:24.658884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 863/1715] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:24.662250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32cdc8-69ee-48b2-95b8-9d4b3551eb4a None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.662561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32cdc8-69ee-48b2-95b8-9d4b3551eb4a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.662845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32cdc8-69ee-48b2-95b8-9d4b3551eb4a None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.668463 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 864/1716] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:42:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:24.670480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32cdc8-69ee-48b2-95b8-9d4b3551eb4a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:24.705712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:24.707235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb681e8653924d51a25f6e1ca4d6fff1', 'request_id': 'req-c92aee37-bb81-498e-b7bd-779b80e5c4ea'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:24.781208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb681e8653924d51a25f6e1ca4d6fff1', 'request_id': 'req-c92aee37-bb81-498e-b7bd-779b80e5c4ea'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:24.788591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb681e8653924d51a25f6e1ca4d6fff1', 'request_id': 'req-c92aee37-bb81-498e-b7bd-779b80e5c4ea'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:24.805213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32cdc8-69ee-48b2-95b8-9d4b3551eb4a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:24.813539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76085da7-c15b-4968-b65c-3445f65423a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:24.815075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085da7-c15b-4968-b65c-3445f65423a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.819116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085da7-c15b-4968-b65c-3445f65423a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.819919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76085da7-c15b-4968-b65c-3445f65423a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.821615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] The token cache is being invalidate because project eb681e8653924d51a25f6e1ca4d6fff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:24.821949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085da7-c15b-4968-b65c-3445f65423a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:24.822643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:24.824900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c92aee37-bb81-498e-b7bd-779b80e5c4ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:24.828627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76085da7-c15b-4968-b65c-3445f65423a3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:24.833549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 853/1717] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:24 2022] DELETE /identity/v3/projects/eb681e8653924d51a25f6e1ca4d6fff1 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:24.838206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.838562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.844061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.860705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 865/1718] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:24.865830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc60591-801e-4ab1-85a0-fb7217fc988f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:24.866184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc60591-801e-4ab1-85a0-fb7217fc988f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:24.866470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc60591-801e-4ab1-85a0-fb7217fc988f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:24.886719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:24.892152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:24.913856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cdc60591-801e-4ab1-85a0-fb7217fc988f None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:24.925247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cdc60591-801e-4ab1-85a0-fb7217fc988f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:24.925370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 854/1719] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:24.926009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:24.927622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:24.970123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc4217c-86ee-4af1-99ce-ddf9d834f0b7 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:24.999487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b527976-5696-4a70-b969-fdbe17aa86a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:25.002342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527976-5696-4a70-b969-fdbe17aa86a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.002703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527976-5696-4a70-b969-fdbe17aa86a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.002946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b527976-5696-4a70-b969-fdbe17aa86a2 admin admin] PATH_INFO: `/v3/projects/6884353dcb8044aab7ba3d007dfe177a/users/91a4dd46dfd74856bb17958b16f8fa3c/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.054496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527976-5696-4a70-b969-fdbe17aa86a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6884353dcb8044aab7ba3d007dfe177a, user_id=91a4dd46dfd74856bb17958b16f8fa3c, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:25.061035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b527976-5696-4a70-b969-fdbe17aa86a2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:25.070915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:25.140458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 866/1720] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:42:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:25.140458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.140458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.140458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.176308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:25.177520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:25.177663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 855/1721] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:24 2022] PUT /identity/v3/projects/6884353dcb8044aab7ba3d007dfe177a/users/91a4dd46dfd74856bb17958b16f8fa3c/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:25.189156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.189780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.190231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.250252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] MFA Rules not processed for user `91a4dd46dfd74856bb17958b16f8fa3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:25.258409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:25.332738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 867/1722] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:25.333729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:25.367360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cb0fa-a8ad-43a1-b29b-cf5f3c5cf4ce None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:25.385487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:25.389907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.390913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.390913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.403360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:25.403739 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:25.410007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:25.473958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 868/1723] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:25.476878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:25.490762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 856/1724] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:25.492326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:25.507791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b0ebe-f552-4bed-80e7-290dab792f42 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:25.517631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8798b7-cab3-4f69-8803-2b56d838805c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:25.539823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:25.541043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:25.541707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.542513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.542782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.542885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.543727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:25.545843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:25.546496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.546875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.548723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:25.553547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:25.725555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 857/1725] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:25 2022] POST /identity/v3/projects => generated 379 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:25.726558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:25.729074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:25.731221 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 869/1726] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:25 2022] POST /identity/v3/projects => generated 387 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:25.732173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:25.899518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679405fb-a163-411e-9880-c5d383962d35 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:25.926774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:25.927399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:25.927706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:25.931703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:25.932158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:25.934122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.067909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 870/1727] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:25 2022] POST /identity/v3/users => generated 296 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:26.068793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:26.076964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692059ed-a789-4907-a603-28e7c4d18148 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:26.094744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db540a3-01c6-4731-b74a-16e723090bc8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:26.106423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536812d-b62d-4cae-bcf2-a6bcf4284f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:26.107107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536812d-b62d-4cae-bcf2-a6bcf4284f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.107388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536812d-b62d-4cae-bcf2-a6bcf4284f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.107680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536812d-b62d-4cae-bcf2-a6bcf4284f75 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.108401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536812d-b62d-4cae-bcf2-a6bcf4284f75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:26.110316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536812d-b62d-4cae-bcf2-a6bcf4284f75 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.244779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:26.255323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 871/1728] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:26 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:26.256929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa7be5-0a49-4355-93af-d39d6676458d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.257359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa7be5-0a49-4355-93af-d39d6676458d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.257501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa7be5-0a49-4355-93af-d39d6676458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.300640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:26.302010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.303570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.304075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.345635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:26.349194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.367371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:26.369024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1ffa7be5-0a49-4355-93af-d39d6676458d None None] MFA Rules not processed for user `a47ab3ebbc9a4a429e5df2a616f305f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:26.372214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1ffa7be5-0a49-4355-93af-d39d6676458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:26.376449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 858/1729] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:26.389851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:26.428781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666ad92-d8f6-4a5c-8ef2-351b2fddf107 service nova] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:26.452778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:26.453677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.455657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.456035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.456750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:26.458523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.502443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 859/1730] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:26.505726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:26.548355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01131c0e-9543-4b5c-88f2-2bd145d00284 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:26.575973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:26.577080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.577727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.578922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.583111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:26.589446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.611456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:26.646710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 872/1731] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:26.650338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafd289-7e5c-4b43-a80f-93f2b43a92b6 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.650338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafd289-7e5c-4b43-a80f-93f2b43a92b6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.650338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafd289-7e5c-4b43-a80f-93f2b43a92b6 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.657370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 860/1732] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:26.657370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:26.661461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 873/1733] 10.222.0.41 () {54 vars in 912 bytes} [Mon Jun 6 20:42:26 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:26.662715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafd289-7e5c-4b43-a80f-93f2b43a92b6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:26.714610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d6cde-2a03-4ead-80a1-d7ce53cb045b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:26.725721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafd289-7e5c-4b43-a80f-93f2b43a92b6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:26.742245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:26.743994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.745141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.745393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] PATH_INFO: `/v3/projects/9f86990085704d0f99f738518d38954d/users/aec4b3131b9c4ebcbac1bc1931ee8934/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.746636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb620023-4576-46d0-bac5-27fa3d22c17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:26.748126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb620023-4576-46d0-bac5-27fa3d22c17f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.749937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb620023-4576-46d0-bac5-27fa3d22c17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.750320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb620023-4576-46d0-bac5-27fa3d22c17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.756758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb620023-4576-46d0-bac5-27fa3d22c17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:26.758981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb620023-4576-46d0-bac5-27fa3d22c17f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.780829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 874/1734] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:26.782652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:26.783154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:26.783711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:26.817001 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f86990085704d0f99f738518d38954d, user_id=aec4b3131b9c4ebcbac1bc1931ee8934, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:26.827422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:26.827860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:26.829596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:26.917179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 861/1735] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:26 2022] PUT /identity/v3/projects/9f86990085704d0f99f738518d38954d/users/aec4b3131b9c4ebcbac1bc1931ee8934/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:26.918284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:27.002020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 875/1736] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:42:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:27.004280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:27.028126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851dce08-76a6-4d2f-8d0b-2f909255b7f6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:27.108641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae35e0-0113-4a53-afc9-e0729e742114 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:27.122048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:27.122953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:27.123239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:27.123521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] PATH_INFO: `/v3/projects/9f1647d3b6c248a2a8c5dda53529df2d/users/9efcfae6216f46a4a9622c17b396f3cf/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:27.151032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:27.154934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:27.155472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:27.156051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:27.157109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:27.162655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:27.169196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1647d3b6c248a2a8c5dda53529df2d, user_id=9efcfae6216f46a4a9622c17b396f3cf, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:27.182373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:27.194587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 876/1737] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:27.196528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:27.205225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:27.273769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 862/1738] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:26 2022] PUT /identity/v3/projects/9f1647d3b6c248a2a8c5dda53529df2d/users/9efcfae6216f46a4a9622c17b396f3cf/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:27.274745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:27.407645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a58b8-aab3-43b2-be1b-4b8c8d78d3c1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:27.553904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79bbdf-cc23-412e-9e95-af4217178053 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:27.583640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbeb07-dcd2-4311-a42b-02ce3896e15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:27.584703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbeb07-dcd2-4311-a42b-02ce3896e15c admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:27.585369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbeb07-dcd2-4311-a42b-02ce3896e15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:27.585803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbeb07-dcd2-4311-a42b-02ce3896e15c admin admin] PATH_INFO: `/v3/projects/9f86990085704d0f99f738518d38954d/users/aec4b3131b9c4ebcbac1bc1931ee8934/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:27.601419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbeb07-dcd2-4311-a42b-02ce3896e15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f86990085704d0f99f738518d38954d, user_id=aec4b3131b9c4ebcbac1bc1931ee8934, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:27.602442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbeb07-dcd2-4311-a42b-02ce3896e15c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:27.618976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:27.622177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:27.623425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:27.624082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:27.624920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:27.632124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 863/1739] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:27 2022] PUT /identity/v3/projects/9f86990085704d0f99f738518d38954d/users/aec4b3131b9c4ebcbac1bc1931ee8934/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:27.633454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebc1b2-e2d0-4207-ac84-a91e7b3e23e1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:27.633754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebc1b2-e2d0-4207-ac84-a91e7b3e23e1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:27.634029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebc1b2-e2d0-4207-ac84-a91e7b3e23e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:27.641366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:27.642984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:27.676698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 877/1740] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:27.677624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:27.693651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:27.731764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c3ebc1b2-e2d0-4207-ac84-a91e7b3e23e1 None None] MFA Rules not processed for user `9efcfae6216f46a4a9622c17b396f3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:27.737670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c3ebc1b2-e2d0-4207-ac84-a91e7b3e23e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:27.976930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 864/1741] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:27.989035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:27.999021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:27.999021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:27.999229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c86ea7-37ae-4fdf-bc9a-2bc7e661da73 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:28.048622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:28.066312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] MFA Rules not processed for user `aec4b3131b9c4ebcbac1bc1931ee8934`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:28.067609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:28.289264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:28.290000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:28.290294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:28.290576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:28.305727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 865/1742] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:28.310064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:28.353298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:28.429775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:28.432388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:28.450035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 878/1743] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 773 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:28.451613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:28.453625 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:28.501953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:28.574294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed346c-4538-490e-85bc-6f744ea11055 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:28.587934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e356c8-6369-4f51-9fe9-0b1f8a3cfa62 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:28.601121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:28.603123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:28.604192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:28.605549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:28.605735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:28.608632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:28.608983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:28.609280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:28.610881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:28.616846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:28.634821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:28.639600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:28.677279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 879/1744] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:28.705449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:28.768971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 866/1745] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:28 2022] POST /identity/v3/projects => generated 385 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:28.771520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:28.798801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e55943-2dcf-495b-bb4d-8ae613b70970 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:28.966845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:28.967816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:28.968954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:28.969409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:28.970298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:28.973283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:29.024810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:29.026734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:29.097198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 880/1746] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:28 2022] POST /identity/v3/users => generated 299 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:29.101017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:29.134543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211d9e8-b34b-4543-8705-1eaa81022d5f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:29.151253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdef6f-dd30-41b8-9be4-24952c83eb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:29.151996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdef6f-dd30-41b8-9be4-24952c83eb39 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.152285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdef6f-dd30-41b8-9be4-24952c83eb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.154038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdef6f-dd30-41b8-9be4-24952c83eb39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:29.154038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdef6f-dd30-41b8-9be4-24952c83eb39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:29.161791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdef6f-dd30-41b8-9be4-24952c83eb39 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:29.202793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 881/1747] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:29.209986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.211813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.212443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:29.244226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c6164-7ee6-4a71-a8d7-597c19a0c8ab admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:29.256109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:29.257430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.257430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.257430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:29.259092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] MFA Rules not processed for user `a10ffb60631240fdb6b4a69d6d7f73e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:29.265666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:29.423703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:29.428677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:29.457511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 867/1748] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:29.458783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:29.499031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c825767-0643-47f2-91b3-ace4ce666521 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:29.514672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:29.517298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.519964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.520388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] PATH_INFO: `/v3/projects/d3ba3ff1e4dd4f95aa68303d2bf2c8e6/users/c5378c43a606499d8d7e9bf5b9951125/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:29.563022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ba3ff1e4dd4f95aa68303d2bf2c8e6, user_id=c5378c43a606499d8d7e9bf5b9951125, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:29.573153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:29.646302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 868/1749] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:29 2022] PUT /identity/v3/projects/d3ba3ff1e4dd4f95aa68303d2bf2c8e6/users/c5378c43a606499d8d7e9bf5b9951125/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:29.650115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:29.657088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 882/1750] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:29.810394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:29.817308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac14c8-6a59-4e17-a1ba-cd05910962b7 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:29.835391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a26fc4-5e82-4f89-9f71-68a3c3112f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:29.836214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a26fc4-5e82-4f89-9f71-68a3c3112f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.836674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a26fc4-5e82-4f89-9f71-68a3c3112f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.837031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a26fc4-5e82-4f89-9f71-68a3c3112f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:29.838292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a26fc4-5e82-4f89-9f71-68a3c3112f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:29.842441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a26fc4-5e82-4f89-9f71-68a3c3112f45 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:29.861166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 869/1751] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:29 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:29.865751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:29.886686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9fa71-27aa-4949-bf6f-6f5e14a3cec5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:29.893758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:29.895023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.895689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.896082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] PATH_INFO: `/v3/projects/d3ba3ff1e4dd4f95aa68303d2bf2c8e6/users/c5378c43a606499d8d7e9bf5b9951125/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:29.899314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ba3ff1e4dd4f95aa68303d2bf2c8e6, user_id=c5378c43a606499d8d7e9bf5b9951125, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:29.903091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:29.944509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 883/1752] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:29 2022] PUT /identity/v3/projects/d3ba3ff1e4dd4f95aa68303d2bf2c8e6/users/c5378c43a606499d8d7e9bf5b9951125/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:29.950808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:29.951186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:29.951540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:30.022737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] MFA Rules not processed for user `c5378c43a606499d8d7e9bf5b9951125`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:30.023895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:30.229986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 870/1753] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:30.240327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:30.360688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:30.398903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8e8093-e836-4817-b485-a840532109fa admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:30.419526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c888f0-4481-477c-8801-586ee0cf5e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:30.423951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c888f0-4481-477c-8801-586ee0cf5e44 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:30.424310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c888f0-4481-477c-8801-586ee0cf5e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:30.427440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c888f0-4481-477c-8801-586ee0cf5e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:30.463465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c888f0-4481-477c-8801-586ee0cf5e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:30.468578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c888f0-4481-477c-8801-586ee0cf5e44 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:30.503616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 884/1754] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:30.558089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:30.588518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:30.589758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:30.590144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:30.704228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] MFA Rules not processed for user `66260219b02d418791a99cd1c9049b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:30.711574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:30.741684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b369f5-ea2b-41df-bfdb-6b73f3b1cad5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:30.754983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:30.760196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:30.761273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:30.761665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] PATH_INFO: `/v3/users/9efcfae6216f46a4a9622c17b396f3cf` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:30.764440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9efcfae6216f46a4a9622c17b396f3cf)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:30.766624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:30.959576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 885/1755] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:30 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:30.960612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:30.965658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:31.000844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ea663b-0d97-4ff5-b3ef-7f9a87024472 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:31.012184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9efcfae6216f46a4a9622c17b396f3cf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:31.016050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9efcfae6216f46a4a9622c17b396f3cf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:31.017054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:31.019547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:31.019893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:31.020303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:31.024243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:31.026219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:31.093630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9efcfae6216f46a4a9622c17b396f3cf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:31.102095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9efcfae6216f46a4a9622c17b396f3cf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:31.154117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:31.158385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1517da18-2628-4987-88dd-d85ec61d9ec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9efcfae6216f46a4a9622c17b396f3cf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:31.178995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 871/1756] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:30 2022] DELETE /identity/v3/users/9efcfae6216f46a4a9622c17b396f3cf => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:31.190575 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 886/1757] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:30 2022] POST /identity/v3/projects => generated 403 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:31.197455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:31.197997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:31.198196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:31.199114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:31.212912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:31.278729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:31.288896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:31.522704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7865d61-6ea6-4e42-acdc-9fae485d02c0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:31.546243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:31.550632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:31.550932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:31.551212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:31.562451 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:31.564466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:31.572428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 872/1758] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:31.700094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 887/1759] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:31 2022] POST /identity/v3/users => generated 308 bytes in 503 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:31.704334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:31.723935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3d83c-debd-4bb7-8236-1cd16f278e76 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:31.754418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:31.758722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:31.759206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:31.761196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:31.763535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:31.778767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:31.781235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:31.808128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:31.889338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 873/1760] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:31.892845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:31.921210 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:31.952076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286abcb9-c68e-41c4-b326-28bdab8ada60 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:31.970373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580400fc-569e-4fdc-8d61-a28963f144d3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:31.981929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce72c15-d321-46ab-8037-a72c9ea09736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:31.986223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce72c15-d321-46ab-8037-a72c9ea09736 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:31.989512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce72c15-d321-46ab-8037-a72c9ea09736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:31.989512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce72c15-d321-46ab-8037-a72c9ea09736 admin admin] PATH_INFO: `/v3/projects/6cfeb6ad18af4127b84e4abaa59f33c0/users/8973b9f860ef408b88c066d436db4fb3/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:32.006222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:32.006990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:32.007379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:32.007559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:32.052701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce72c15-d321-46ab-8037-a72c9ea09736 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfeb6ad18af4127b84e4abaa59f33c0, user_id=8973b9f860ef408b88c066d436db4fb3, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:32.060669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:32.065592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:32.065790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce72c15-d321-46ab-8037-a72c9ea09736 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:32.112364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 888/1761] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:32.232603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 874/1762] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:31 2022] PUT /identity/v3/projects/6cfeb6ad18af4127b84e4abaa59f33c0/users/8973b9f860ef408b88c066d436db4fb3/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:32.233983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:32.253816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a1f02-8324-4c47-af09-c63ba4890e9b service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:32.382246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:32.382563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:32.382981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:32.483388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] MFA Rules not processed for user `3d3896bd70dd43fe95f7125f9f4693a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:32.488072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:32.622707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:32.627119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:32.629289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:32.630357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:32.638448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:32.638448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:32.686293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 889/1763] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 452 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:32.686447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:32.723943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd89afb-0f62-4af4-a761-809ba8c64d52 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:32.743702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:32.749993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 875/1764] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:32.754486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:32.755287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:32.755577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] PATH_INFO: `/v3/projects/6cfeb6ad18af4127b84e4abaa59f33c0/users/8973b9f860ef408b88c066d436db4fb3/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:32.758077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfeb6ad18af4127b84e4abaa59f33c0, user_id=8973b9f860ef408b88c066d436db4fb3, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:32.763730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:32.776677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 890/1765] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:32 2022] PUT /identity/v3/projects/6cfeb6ad18af4127b84e4abaa59f33c0/users/8973b9f860ef408b88c066d436db4fb3/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:33.902611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:33.904110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:33.931969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3bad2-329d-4bf8-a658-5ab8edb5193d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:34.035309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:34.036650 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:34.036942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:34.039147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] PATH_INFO: `/v3/projects/9f1647d3b6c248a2a8c5dda53529df2d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:34.042414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1647d3b6c248a2a8c5dda53529df2d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:34.047210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:34.070969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:34.216161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1647d3b6c248a2a8c5dda53529df2d', 'request_id': 'req-f8860c49-2941-4731-8b2a-e6be25845bb7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.320937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1647d3b6c248a2a8c5dda53529df2d', 'request_id': 'req-f8860c49-2941-4731-8b2a-e6be25845bb7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.336222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1647d3b6c248a2a8c5dda53529df2d', 'request_id': 'req-f8860c49-2941-4731-8b2a-e6be25845bb7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.358902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] The token cache is being invalidate because project 9f1647d3b6c248a2a8c5dda53529df2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:34.359436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.360413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.366298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 876/1766] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:33 2022] DELETE /identity/v3/projects/9f1647d3b6c248a2a8c5dda53529df2d => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:34.367059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:34.368918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:34.383623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cd9006-648b-4cd0-82a3-490fe6583745 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:34.517254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:34.518778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:34.519091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:34.519338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] PATH_INFO: `/v3/users/c5378c43a606499d8d7e9bf5b9951125` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:34.521819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5378c43a606499d8d7e9bf5b9951125)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:34.523975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:34.813097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5378c43a606499d8d7e9bf5b9951125'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.813821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5378c43a606499d8d7e9bf5b9951125'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.877575 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5378c43a606499d8d7e9bf5b9951125'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.897678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:34.909442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5378c43a606499d8d7e9bf5b9951125'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.918585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-297e6be6-1676-46dc-99ec-ed3d25c23c30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5378c43a606499d8d7e9bf5b9951125'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:34.925505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 891/1767] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:34 2022] DELETE /identity/v3/users/c5378c43a606499d8d7e9bf5b9951125 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:35.473411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:35.477517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c37e-e39a-487e-9dfc-213c1b6ff453 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:35.478068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c37e-e39a-487e-9dfc-213c1b6ff453 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:35.479426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e970c37e-e39a-487e-9dfc-213c1b6ff453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:35.492337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860c49-2941-4731-8b2a-e6be25845bb7 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:35.513910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e970c37e-e39a-487e-9dfc-213c1b6ff453 None None] MFA Rules not processed for user `a47ab3ebbc9a4a429e5df2a616f305f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:35.515216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e970c37e-e39a-487e-9dfc-213c1b6ff453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:35.606430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:35.607934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:35.608934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:35.609371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] PATH_INFO: `/v3/projects/d3ba3ff1e4dd4f95aa68303d2bf2c8e6` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:35.613427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ba3ff1e4dd4f95aa68303d2bf2c8e6)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:35.617110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:35.638066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 892/1768] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:35.681577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b5f1-a282-4cd0-bfe0-aaf436254e45 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:35.682888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b5f1-a282-4cd0-bfe0-aaf436254e45 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:35.683882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2356b5f1-a282-4cd0-bfe0-aaf436254e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:35.747808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2356b5f1-a282-4cd0-bfe0-aaf436254e45 None None] MFA Rules not processed for user `8973b9f860ef408b88c066d436db4fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:35.749430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2356b5f1-a282-4cd0-bfe0-aaf436254e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:35.755797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ba3ff1e4dd4f95aa68303d2bf2c8e6', 'request_id': 'req-bf203e68-0ad1-493e-99f5-bc8d8f679074'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:35.830355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ba3ff1e4dd4f95aa68303d2bf2c8e6', 'request_id': 'req-bf203e68-0ad1-493e-99f5-bc8d8f679074'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:35.837212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ba3ff1e4dd4f95aa68303d2bf2c8e6', 'request_id': 'req-bf203e68-0ad1-493e-99f5-bc8d8f679074'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:35.857631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] The token cache is being invalidate because project d3ba3ff1e4dd4f95aa68303d2bf2c8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:35.857929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:35.858721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf203e68-0ad1-493e-99f5-bc8d8f679074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:35.859445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 893/1769] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:35 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:35.865410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 877/1770] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:35 2022] DELETE /identity/v3/projects/d3ba3ff1e4dd4f95aa68303d2bf2c8e6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:35.871151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:35.871558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:35.872572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:35.898690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:35.899904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:36.000943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.001324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.001680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.003755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 894/1771] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:36.043103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:36.044413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:36.068486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 878/1772] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:36.070666 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.087242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf43502-63fd-4ab1-991d-50a4192381ff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.092803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.093983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.094349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.094638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.095698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:36.095959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.097616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.110491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:36.124139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 895/1773] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:36.125285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.136291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e40068-2e72-4049-86e9-9234eb3525d3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.145994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.146890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.147255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.147482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.148407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.150867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.243248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:36.252718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 879/1774] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:36 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:36.256607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.268806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0dd5d-9c0b-455a-a159-a50c1eb0e2fb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.348396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.349257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.349664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.350018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.350991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.353057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.445805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 896/1775] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:36 2022] POST /identity/v3/users => generated 294 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:36.447093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.457812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10784cd5-c6bf-49da-b1fe-64d3cb5c6d48 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.462313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.462946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.463232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.464030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.464982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.466615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.477490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 880/1776] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:36.482082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.495905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43bdb4-b038-407f-94b0-5b6dd84a2e59 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.503203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377dc2-3bdb-45a0-9239-b6971350303f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.504256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377dc2-3bdb-45a0-9239-b6971350303f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.504878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377dc2-3bdb-45a0-9239-b6971350303f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.505456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377dc2-3bdb-45a0-9239-b6971350303f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.506427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377dc2-3bdb-45a0-9239-b6971350303f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.508774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377dc2-3bdb-45a0-9239-b6971350303f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.525273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 897/1777] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:36.528471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.539114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe916f-46b2-48b3-afd0-061a848501de admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.544393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236eb89-e842-40c7-b413-77bd12b49f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.545268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236eb89-e842-40c7-b413-77bd12b49f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.545582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236eb89-e842-40c7-b413-77bd12b49f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.545860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236eb89-e842-40c7-b413-77bd12b49f0a admin admin] PATH_INFO: `/v3/projects/c681ba30e8d14965ba49b0a32656be07/users/5101278bd46a488cad55d1814d1b19ed/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.560518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236eb89-e842-40c7-b413-77bd12b49f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c681ba30e8d14965ba49b0a32656be07, user_id=5101278bd46a488cad55d1814d1b19ed, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.563152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236eb89-e842-40c7-b413-77bd12b49f0a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.638810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 881/1778] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:36 2022] PUT /identity/v3/projects/c681ba30e8d14965ba49b0a32656be07/users/5101278bd46a488cad55d1814d1b19ed/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:36.650444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.650729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.651646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.680715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] MFA Rules not processed for user `5101278bd46a488cad55d1814d1b19ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:36.681932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:36.757617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef513-cb3c-40fa-a4ab-234a45939ba0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.757915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef513-cb3c-40fa-a4ab-234a45939ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.758995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef513-cb3c-40fa-a4ab-234a45939ba0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.760340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 882/1779] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:42:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:36.772197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.773268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.779342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.779853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 898/1780] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:36.779853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.802757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:36.804746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:36.926151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 883/1781] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:42:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:36.932864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a1a00-138d-4ec7-a93f-70aeb37f7ccb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.941515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.942727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.947783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.947783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.947783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.947783 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:36.951465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:36.953121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:36.964694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:36.973662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39ede8-21d1-4c40-8f50-4d5dd26ddac2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:36.979899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:36.980559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:36.980901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:36.981188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:36.992741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:36.994537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:37.006324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 884/1782] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:37.090334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 899/1783] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:36 2022] POST /identity/v3/projects => generated 387 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:37.094417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.105171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3f2b7-0f86-4835-b7f2-d3881bcc92de service placement] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:37.111591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.113447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:37.205079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:37.205780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.206073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.206358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.207113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:37.209407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:37.222140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794881b0-8e2a-4a7f-8109-755f1d58b251 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:37.240119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:37.241283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.245718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.245718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] PATH_INFO: `/v3/users/b804c67483e34dcab9a0fedab8388b27` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.245718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b804c67483e34dcab9a0fedab8388b27)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:37.247332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:37.314630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 885/1784] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:37 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:37.316032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.328326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0041e9-bb69-49fa-9f8c-e7c682470c80 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:37.338312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:37.339096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.339398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.339685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.340580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:37.342481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:37.358654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 886/1785] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:37.360079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.371107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5277c-6bb1-4219-bf1b-db722030e300 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:37.375954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:37.376627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.376913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.377188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.377913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:37.379602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:37.391916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 887/1786] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:37.396267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.407271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee942c-ef66-44c8-9ed9-5332a23469c9 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:37.414705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e7c59-63f2-47e9-be25-22904c2c9392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:37.421722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e7c59-63f2-47e9-be25-22904c2c9392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.421722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e7c59-63f2-47e9-be25-22904c2c9392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.421722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e7c59-63f2-47e9-be25-22904c2c9392 admin admin] PATH_INFO: `/v3/projects/ab9df7e19d084a7885062171ddad1488/users/1f97b72334744d82b08bff082e9cf686/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.432941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:37.437837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e7c59-63f2-47e9-be25-22904c2c9392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9df7e19d084a7885062171ddad1488, user_id=1f97b72334744d82b08bff082e9cf686, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:37.443925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e7c59-63f2-47e9-be25-22904c2c9392 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:37.481737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b804c67483e34dcab9a0fedab8388b27'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:37.482466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b804c67483e34dcab9a0fedab8388b27'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:37.517371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 888/1787] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:37 2022] PUT /identity/v3/projects/ab9df7e19d084a7885062171ddad1488/users/1f97b72334744d82b08bff082e9cf686/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:37.528366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c73560-49f7-4763-9451-52793832af81 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.528652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c73560-49f7-4763-9451-52793832af81 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.529131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c73560-49f7-4763-9451-52793832af81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.560154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d1c73560-49f7-4763-9451-52793832af81 None None] MFA Rules not processed for user `1f97b72334744d82b08bff082e9cf686`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:37.561446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d1c73560-49f7-4763-9451-52793832af81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:37.579085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b804c67483e34dcab9a0fedab8388b27'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:37.593548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b804c67483e34dcab9a0fedab8388b27'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:37.646363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 889/1788] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:37.660372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e986417e-7332-4b21-8085-9a2e9ee8279c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b804c67483e34dcab9a0fedab8388b27'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:37.661160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c73560-49f7-4763-9451-52793832af81 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.664883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d1c73560-49f7-4763-9451-52793832af81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:37.695989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 900/1789] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:37 2022] DELETE /identity/v3/users/b804c67483e34dcab9a0fedab8388b27 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:37.700897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.701186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.701510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.753831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:37.757323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:37.771906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c73560-49f7-4763-9451-52793832af81 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:37.857360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 901/1790] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:37.886729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:37.888501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:37.939164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:37.958317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6b0a2-9d5a-45c6-8c15-e929b27eada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:37.958952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b0a2-9d5a-45c6-8c15-e929b27eada3 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:37.959630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b0a2-9d5a-45c6-8c15-e929b27eada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:37.959762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6b0a2-9d5a-45c6-8c15-e929b27eada3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:37.960388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6b0a2-9d5a-45c6-8c15-e929b27eada3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:37.965990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6b0a2-9d5a-45c6-8c15-e929b27eada3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.067561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 890/1791] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:37 2022] POST /identity/v3/projects => generated 385 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:38.069044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43b119-ee3f-4d63-804b-dd8c925b91eb None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.069446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43b119-ee3f-4d63-804b-dd8c925b91eb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.071470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f43b119-ee3f-4d63-804b-dd8c925b91eb None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.080720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 891/1792] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:42:38 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:38.081784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43b119-ee3f-4d63-804b-dd8c925b91eb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:38.083321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281c506-30f5-4217-a4ad-6804644da6ff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.135311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:38.269635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a2a8e-ba1a-4f0d-8fd9-e9eaa9afd3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.269864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2a8e-ba1a-4f0d-8fd9-e9eaa9afd3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.275764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2a8e-ba1a-4f0d-8fd9-e9eaa9afd3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.276088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2a8e-ba1a-4f0d-8fd9-e9eaa9afd3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.294513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f43b119-ee3f-4d63-804b-dd8c925b91eb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.302128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.303315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.303729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.304138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.304938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:38.306975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.315035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a2a8e-ba1a-4f0d-8fd9-e9eaa9afd3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:38.315899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a2a8e-ba1a-4f0d-8fd9-e9eaa9afd3a7 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.329513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 902/1793] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:38.330483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.330773 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.331047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.356538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] MFA Rules not processed for user `4dd69635d9a5418e818de21ca309f9a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:38.358450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:38.434831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 903/1794] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:42:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:38.435746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:38.437401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:38.457009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdab512-b3fe-4d6b-8132-088400271e24 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.463417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.464569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.464868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.465219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] PATH_INFO: `/v3/users/66260219b02d418791a99cd1c9049b74` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.467029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66260219b02d418791a99cd1c9049b74)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:38.469882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.485354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 892/1795] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:38 2022] POST /identity/v3/users => generated 299 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:38.488115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:38.489845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:38.509752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ca0e97-0cf6-4784-9247-8aa59ee9565d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.524775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.525368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.526915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.527269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.538805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:38.540995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.555968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 893/1796] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:38.556999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:38.582540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c06cfcc-1465-453f-b5f0-c4782f4275cd service placement] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.594174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.595228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.595599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.595966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.596878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:38.599772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.612551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 894/1797] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:38.616904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:38.641149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d2af9-55fb-4bf5-85f8-85d7af966c58 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.662785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.665690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.666180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.667482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] PATH_INFO: `/v3/projects/48bec031e7774d6882a0ae5c51f89249/users/eca4ef6385604acabd71c907b3f7b43f/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:38.686239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bec031e7774d6882a0ae5c51f89249, user_id=eca4ef6385604acabd71c907b3f7b43f, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:38.688998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:38.740497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66260219b02d418791a99cd1c9049b74'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:38.741168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66260219b02d418791a99cd1c9049b74'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:38.819326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 895/1798] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:38 2022] PUT /identity/v3/projects/48bec031e7774d6882a0ae5c51f89249/users/eca4ef6385604acabd71c907b3f7b43f/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:38.823239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:38.825024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:38.842537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66260219b02d418791a99cd1c9049b74'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:38.863557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66260219b02d418791a99cd1c9049b74'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:38.897227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66260219b02d418791a99cd1c9049b74'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:38.912189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 904/1799] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:38 2022] DELETE /identity/v3/users/66260219b02d418791a99cd1c9049b74 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:38.924925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de269d-d592-4f77-b7c4-13f8eeaee51f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:38.997078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb7f30-c5cc-42d6-829a-d7ac207ae31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:38.997850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb7f30-c5cc-42d6-829a-d7ac207ae31f admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:38.998204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb7f30-c5cc-42d6-829a-d7ac207ae31f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:38.998600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb7f30-c5cc-42d6-829a-d7ac207ae31f admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:39.000388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb7f30-c5cc-42d6-829a-d7ac207ae31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:39.001828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb7f30-c5cc-42d6-829a-d7ac207ae31f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:39.018517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 896/1800] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:39.019948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:39.047078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8b70b-c93f-402f-9967-38476c62ba2c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:39.059952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:39.060856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:39.061284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:39.061604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] PATH_INFO: `/v3/projects/48bec031e7774d6882a0ae5c51f89249/users/eca4ef6385604acabd71c907b3f7b43f/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:39.066162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=48bec031e7774d6882a0ae5c51f89249, user_id=eca4ef6385604acabd71c907b3f7b43f, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:39.069007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:39.099373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 905/1801] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:39 2022] PUT /identity/v3/projects/48bec031e7774d6882a0ae5c51f89249/users/eca4ef6385604acabd71c907b3f7b43f/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:39.111893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:39.111893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:39.111893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:39.141374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] MFA Rules not processed for user `eca4ef6385604acabd71c907b3f7b43f`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:39.142466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:39.374413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:39.429589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 897/1802] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:39.438828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:39.441896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:39.562319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:39.625308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7942fa-0323-4067-94b7-de0c768b5d1f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:39.634444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:39.635455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:39.635975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:39.636982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:39.648317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:39.650128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:39.673819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 906/1803] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:39 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:39.756659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:39.758254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:39.876028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a2872-f37f-4ccb-99bd-34bfefa59e7b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:39.881488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:39.882756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:39.883208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:39.883622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] PATH_INFO: `/v3/projects/bbba9a6248554e9ab398f3b55c59ce68` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:39.885338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbba9a6248554e9ab398f3b55c59ce68)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:39.888010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:39.905604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:39.907082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:39.946681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d02e60-ec93-4cb8-9db8-0b5dc8eaa58b service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:39.969472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871f83b-721e-4261-824d-416320420776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:39.972402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871f83b-721e-4261-824d-416320420776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:39.972849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871f83b-721e-4261-824d-416320420776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:39.974606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871f83b-721e-4261-824d-416320420776 admin admin] PATH_INFO: `/v3/projects/952b2a2d9302405882681f6ca82b0dde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:39.977769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871f83b-721e-4261-824d-416320420776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952b2a2d9302405882681f6ca82b0dde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:39.980626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871f83b-721e-4261-824d-416320420776 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:40.014937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbba9a6248554e9ab398f3b55c59ce68', 'request_id': 'req-292ddf3d-62cc-4c4d-a595-527531fc6ddd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.114151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbba9a6248554e9ab398f3b55c59ce68', 'request_id': 'req-292ddf3d-62cc-4c4d-a595-527531fc6ddd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.145377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbba9a6248554e9ab398f3b55c59ce68', 'request_id': 'req-292ddf3d-62cc-4c4d-a595-527531fc6ddd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.185936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] The token cache is being invalidate because project bbba9a6248554e9ab398f3b55c59ce68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:40.188046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.190536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-292ddf3d-62cc-4c4d-a595-527531fc6ddd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.195031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952b2a2d9302405882681f6ca82b0dde', 'request_id': 'req-1871f83b-721e-4261-824d-416320420776'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.201934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 898/1804] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:39 2022] DELETE /identity/v3/projects/bbba9a6248554e9ab398f3b55c59ce68 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:40.202961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f3b4-0245-4ed0-ab92-7bde939ea8d0 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:40.203258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f3b4-0245-4ed0-ab92-7bde939ea8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:40.203538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1553f3b4-0245-4ed0-ab92-7bde939ea8d0 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:40.210564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 899/1805] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:42:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:40.211425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f3b4-0245-4ed0-ab92-7bde939ea8d0 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:40.220819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1553f3b4-0245-4ed0-ab92-7bde939ea8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:40.344624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952b2a2d9302405882681f6ca82b0dde', 'request_id': 'req-1871f83b-721e-4261-824d-416320420776'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.354910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952b2a2d9302405882681f6ca82b0dde', 'request_id': 'req-1871f83b-721e-4261-824d-416320420776'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.379824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1871f83b-721e-4261-824d-416320420776 admin admin] The token cache is being invalidate because project 952b2a2d9302405882681f6ca82b0dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:40.380121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.382388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.390219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 907/1806] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:39 2022] DELETE /identity/v3/projects/952b2a2d9302405882681f6ca82b0dde => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:40.391248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:40.394630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:40.469369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1553f3b4-0245-4ed0-ab92-7bde939ea8d0 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:40.485610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:40.491607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:40.495115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:40.495583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] PATH_INFO: `/v3/users/ee5f9d7f29da47658821ddea73df84da` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:40.502461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee5f9d7f29da47658821ddea73df84da)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:40.509532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:40.586314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871f83b-721e-4261-824d-416320420776 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:40.593502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:40.594400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:40.594692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:40.594972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] PATH_INFO: `/v3/users/8973b9f860ef408b88c066d436db4fb3` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:40.596330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8973b9f860ef408b88c066d436db4fb3)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:40.598529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:40.797644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee5f9d7f29da47658821ddea73df84da'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.800054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee5f9d7f29da47658821ddea73df84da'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.886818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8973b9f860ef408b88c066d436db4fb3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.887828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8973b9f860ef408b88c066d436db4fb3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.899162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee5f9d7f29da47658821ddea73df84da'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.908792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee5f9d7f29da47658821ddea73df84da'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.933499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-46992b4d-9836-48a7-9304-8ba28a691791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee5f9d7f29da47658821ddea73df84da'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:40.951139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 900/1807] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:40 2022] DELETE /identity/v3/users/ee5f9d7f29da47658821ddea73df84da => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:40.955313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:40.955313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:40.955519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:41.013135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:41.014471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:41.020014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8973b9f860ef408b88c066d436db4fb3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:41.038746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8973b9f860ef408b88c066d436db4fb3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:41.057032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8973b9f860ef408b88c066d436db4fb3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:41.079458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:41.081921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 908/1808] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:40 2022] DELETE /identity/v3/users/8973b9f860ef408b88c066d436db4fb3 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:41.164625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 901/1809] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:42:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:41.178066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:41.179915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:41.270453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74799a-5c7d-4c72-a56f-a53dd8c9e375 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:41.277568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:41.278311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:41.278608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:41.279336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:41.291198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:41.293448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:41.311220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 909/1810] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:41.597130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:41.599756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:41.692394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f33a60-32c8-4b23-a9f5-f6732d976783 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:41.764973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:41.765971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:41.766442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:41.766900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:41.770090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:41.808487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:41.850357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:41.852068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:41.895297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:41.898770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d01e08-86e8-4bf6-bccd-4b3482dd35c4 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:41.925616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 902/1811] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:41.939189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36fc92-c0f0-4720-9754-7f5bf2052e7c service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:41.944781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:41.946460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:41.946924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:41.947389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] PATH_INFO: `/v3/projects/6cfeb6ad18af4127b84e4abaa59f33c0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:41.949364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfeb6ad18af4127b84e4abaa59f33c0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:41.951494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:42.098984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfeb6ad18af4127b84e4abaa59f33c0', 'request_id': 'req-73d768a5-8f73-4363-b8f9-322ad5dbe368'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.162421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfeb6ad18af4127b84e4abaa59f33c0', 'request_id': 'req-73d768a5-8f73-4363-b8f9-322ad5dbe368'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.172126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfeb6ad18af4127b84e4abaa59f33c0', 'request_id': 'req-73d768a5-8f73-4363-b8f9-322ad5dbe368'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.188282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] The token cache is being invalidate because project 6cfeb6ad18af4127b84e4abaa59f33c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:42.188938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.189766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-73d768a5-8f73-4363-b8f9-322ad5dbe368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.191557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 910/1812] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:41 2022] DELETE /identity/v3/projects/6cfeb6ad18af4127b84e4abaa59f33c0 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:42.196838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.197145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.197415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.219507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:42.222212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:42.329152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 903/1813] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:42.339536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.339856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.340143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.363476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:42.368137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:42.382847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 911/1814] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:42.387992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:42.406095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8170b-4889-436d-b292-48d6382b2518 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:42.414293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:42.415626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.416038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.416423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.418605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:42.419028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:42.421481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:42.438555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 904/1815] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:42.443382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:42.454322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b76bfa-9d1b-4ca5-beee-c83fae70ccd7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:42.460249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:42.465644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.465644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.465644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.465644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:42.465644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:42.467105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:42.470532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:42.490468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b0668-90f4-4c9a-a8bb-b32301dc750a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:42.495548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:42.497236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.497586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.497895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] PATH_INFO: `/v3/projects/cc87dddd9a7440afb8d80ae9e97f9143` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.499255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc87dddd9a7440afb8d80ae9e97f9143)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:42.501886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:42.577737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 912/1816] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:42 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:42:42.577737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:42.579139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:42.646160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc87dddd9a7440afb8d80ae9e97f9143', 'request_id': 'req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.646401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45fa6fa-250a-4467-b676-333831cc16ff admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:42.727493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc87dddd9a7440afb8d80ae9e97f9143', 'request_id': 'req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.739203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc87dddd9a7440afb8d80ae9e97f9143', 'request_id': 'req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.771147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] The token cache is being invalidate because project cc87dddd9a7440afb8d80ae9e97f9143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:42.771436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.773733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:42.778117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 905/1817] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:42 2022] DELETE /identity/v3/projects/cc87dddd9a7440afb8d80ae9e97f9143 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:42.779446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:42.783862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:42.785218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:42.785839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.789262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.789533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] PATH_INFO: `/v3/users/91a4dd46dfd74856bb17958b16f8fa3c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.794239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a4dd46dfd74856bb17958b16f8fa3c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:42.809400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:42.837806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a31f35-c83e-47d2-a60a-66b3abfc5ab0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:42.856813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbffd14-68c5-4b6e-9515-9c84c130fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:42.858592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbffd14-68c5-4b6e-9515-9c84c130fa2d admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:42.858855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbffd14-68c5-4b6e-9515-9c84c130fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:42.859140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbffd14-68c5-4b6e-9515-9c84c130fa2d admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:42.859883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbffd14-68c5-4b6e-9515-9c84c130fa2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:42.864449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbffd14-68c5-4b6e-9515-9c84c130fa2d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.012125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:43.022828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 906/1818] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:42 2022] POST /identity/v3/users => generated 282 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:43.024457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.025184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.025899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.074394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:43.075642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:43.096093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 907/1819] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:43.101272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.109558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb35ff-b920-4e88-9bf2-acbe6acef82a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.117635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc638d8-8527-40cb-a741-6d9256891ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.118483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc638d8-8527-40cb-a741-6d9256891ca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.118884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc638d8-8527-40cb-a741-6d9256891ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.119268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc638d8-8527-40cb-a741-6d9256891ca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.122972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc638d8-8527-40cb-a741-6d9256891ca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.125305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc638d8-8527-40cb-a741-6d9256891ca2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.130759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a4dd46dfd74856bb17958b16f8fa3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:43.137294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a4dd46dfd74856bb17958b16f8fa3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:43.142167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 908/1820] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:43.144991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.145375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.145808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.170014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:43.171343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:43.199231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 909/1821] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:43.201594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.207034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:43.247809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a4dd46dfd74856bb17958b16f8fa3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:43.249143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59075d80-c558-42d3-b94b-ff4c4f4ab4f7 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.264416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.265341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.265341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.266940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.268434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.278424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a4dd46dfd74856bb17958b16f8fa3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:43.280228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.290772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a4dd46dfd74856bb17958b16f8fa3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:43.305145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 910/1822] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:43.308029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 913/1823] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:42 2022] DELETE /identity/v3/users/91a4dd46dfd74856bb17958b16f8fa3c => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:43.313699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.318056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.353391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c3dffa-d50c-4416-b60a-ad9bf7030cc4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.424060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87c5aa-eb1a-420d-b535-d8e9fdca3361 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.430606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f72a9-66a2-4e67-a0ef-085dc3f03e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.431937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f72a9-66a2-4e67-a0ef-085dc3f03e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.432926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f72a9-66a2-4e67-a0ef-085dc3f03e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.433356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f72a9-66a2-4e67-a0ef-085dc3f03e3c admin admin] PATH_INFO: `/v3/projects/70e65584be1f41dbbff58d9027d51213/users/ab5809ffcd2244c68e3f831594cef9d1/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.455545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f72a9-66a2-4e67-a0ef-085dc3f03e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e65584be1f41dbbff58d9027d51213, user_id=ab5809ffcd2244c68e3f831594cef9d1, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.459843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f72a9-66a2-4e67-a0ef-085dc3f03e3c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.475765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.480604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.480604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.480604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.480604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:43.480604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.481915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0bd63-7634-4d08-8d8c-fedbf960dfa0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.504893 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 911/1824] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:43.507749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.508263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.508821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.530469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:43.532519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:43.551449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 912/1825] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:43.552501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.567649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe62463-5d13-4d6f-a9d3-01ba2ed1baff None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.569941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 914/1826] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:43 2022] PUT /identity/v3/projects/70e65584be1f41dbbff58d9027d51213/users/ab5809ffcd2244c68e3f831594cef9d1/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:43.583942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.584425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.585131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.628433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] MFA Rules not processed for user `ab5809ffcd2244c68e3f831594cef9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:43.629843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:43.644532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.645674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.646381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.646782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.647913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.650206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.718690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 915/1827] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:43.720494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.722920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:43.749611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 913/1828] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:43 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:43.750344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:43.780978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:43.883137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85c26b-6b3b-4fa4-80e9-8ef6e89aef7c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.903496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6755a8c-b8ca-4895-8ca4-7371ce0297b2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:43.934767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.935450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.935784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.936060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.936757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.941759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:43.966353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:43.967211 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:43.967510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:43.968406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:43.984270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:43.988351 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.020073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 916/1829] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:44.020857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.034874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39809780-45fb-4558-8787-9a450787ce9a service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.046179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.046870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.047158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.047434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.048162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.050094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.081025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 914/1830] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:43 2022] POST /identity/v3/projects => generated 363 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:44.090083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.162466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 917/1831] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:44 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:44.167567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.184546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31c5292-e8e6-4621-9146-7227bad16a8d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.198213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b8e59-907f-4108-8ac4-b0304b06a0db admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.209354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.210034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.210380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.210661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.211408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.214108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.215273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.216164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.216768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.216768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.217963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.223438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.237838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 918/1832] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:44.244969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.259322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57cf7-d258-4246-8bb3-abe1a6cba4d3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.269787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.270547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.274112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.274112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.274112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.274112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.290455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 919/1833] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:44.297255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.314361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f016b1-ed3c-4d75-9f0b-b7ce8cdee003 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.317577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800b41b-1126-4dd1-9c6d-f3ff6577a9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.318983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b41b-1126-4dd1-9c6d-f3ff6577a9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.319270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b41b-1126-4dd1-9c6d-f3ff6577a9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.319552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800b41b-1126-4dd1-9c6d-f3ff6577a9cd admin admin] PATH_INFO: `/v3/projects/aeb3e08cf36d4f5aad0fb6de90194bce/users/f58674cb6ccf46bba015872948016762/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.340183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800b41b-1126-4dd1-9c6d-f3ff6577a9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeb3e08cf36d4f5aad0fb6de90194bce, user_id=f58674cb6ccf46bba015872948016762, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.346752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800b41b-1126-4dd1-9c6d-f3ff6577a9cd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.358715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 915/1834] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:44 2022] POST /identity/v3/users => generated 288 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:44.363609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.375528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9523d73-f139-4430-b8da-05cc6d78fcd4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.388751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.389899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.389899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.390363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.391148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.392811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.409926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 916/1835] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:44.411174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.426061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f7811-34e7-4423-8d62-403dac9f1c3b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.434594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.436250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.443699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.452399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] PATH_INFO: `/v3/projects/7211b0a2321a4354bdabccbadc38c1f2/users/2fac8dc2d1f34a9b83e78e12ded548e6/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.479220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7211b0a2321a4354bdabccbadc38c1f2, user_id=2fac8dc2d1f34a9b83e78e12ded548e6, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.482335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 920/1836] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:44 2022] PUT /identity/v3/projects/aeb3e08cf36d4f5aad0fb6de90194bce/users/f58674cb6ccf46bba015872948016762/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:44.492535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.492828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.493108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.494019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.550500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] MFA Rules not processed for user `f58674cb6ccf46bba015872948016762`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:44.553483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:44.618063 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 917/1837] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:44 2022] PUT /identity/v3/projects/7211b0a2321a4354bdabccbadc38c1f2/users/2fac8dc2d1f34a9b83e78e12ded548e6/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:44.619038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.649391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 921/1838] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:44.655311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.709802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a525f0-363c-498a-bc19-c43df7eefc72 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.715239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.716275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.716759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.717213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.718125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.719971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.734979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 918/1839] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:44.736215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:44.746011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:44.753219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0807bd6-c6f9-4348-a618-507536059d59 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.759528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c56cb8-17c6-45b3-93f0-3058f61018f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.761685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c56cb8-17c6-45b3-93f0-3058f61018f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.762739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c56cb8-17c6-45b3-93f0-3058f61018f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.763153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c56cb8-17c6-45b3-93f0-3058f61018f0 admin admin] PATH_INFO: `/v3/projects/7211b0a2321a4354bdabccbadc38c1f2/users/2fac8dc2d1f34a9b83e78e12ded548e6/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.765432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c56cb8-17c6-45b3-93f0-3058f61018f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7211b0a2321a4354bdabccbadc38c1f2, user_id=2fac8dc2d1f34a9b83e78e12ded548e6, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.766846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb577a5c-e2df-4ad8-b883-e36e7507cb4a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:44.768958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c56cb8-17c6-45b3-93f0-3058f61018f0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.783487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:44.784518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.787398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.787398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.787398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:44.790719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:44.797468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 919/1840] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:44 2022] PUT /identity/v3/projects/7211b0a2321a4354bdabccbadc38c1f2/users/2fac8dc2d1f34a9b83e78e12ded548e6/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:44.799779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:44.800173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:44.800754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:44.851067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] MFA Rules not processed for user `2fac8dc2d1f34a9b83e78e12ded548e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:44.852941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:44.959149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 922/1841] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:44 2022] POST /identity/v3/projects => generated 367 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:44.960094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.020498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 920/1842] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:44 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:45.032422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.034130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:45.088542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b016bd10-8242-45d6-97da-594e56d51b45 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:45.095235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f10ce7-4c23-4677-ac73-0b56a487e193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:45.096301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f10ce7-4c23-4677-ac73-0b56a487e193 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.096786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f10ce7-4c23-4677-ac73-0b56a487e193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.097254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f10ce7-4c23-4677-ac73-0b56a487e193 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.098228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f10ce7-4c23-4677-ac73-0b56a487e193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:45.100729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f10ce7-4c23-4677-ac73-0b56a487e193 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:45.154838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e30a871-ed1b-48bf-bd31-5cddd8008a5c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:45.227385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 923/1843] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:44 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:45.228705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469935d-cbbb-4d2b-a6bf-e10547b3a2e2 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.235931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469935d-cbbb-4d2b-a6bf-e10547b3a2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.236303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469935d-cbbb-4d2b-a6bf-e10547b3a2e2 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.242098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 924/1844] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:42:45 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:42:45.242785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469935d-cbbb-4d2b-a6bf-e10547b3a2e2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.294163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469935d-cbbb-4d2b-a6bf-e10547b3a2e2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:45.318802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:45.320903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6721adb-8c24-4b89-82f0-777c181a4149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:45.321844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6721adb-8c24-4b89-82f0-777c181a4149 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.322051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6721adb-8c24-4b89-82f0-777c181a4149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.322334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6721adb-8c24-4b89-82f0-777c181a4149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.323106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6721adb-8c24-4b89-82f0-777c181a4149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:45.324912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6721adb-8c24-4b89-82f0-777c181a4149 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:45.353279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 925/1845] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:45.355446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.355798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.357228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:45.357882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.358164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.358598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.360905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.408639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:45.409880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:45.501923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:45.505231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:45.529663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 921/1846] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:45.530772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.546999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fbbd74-6b7d-400f-8e94-f94fe3cc2a87 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:45.552727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:45.553872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.554527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.554811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] PATH_INFO: `/v3/projects/ebb8e43bfc36450aa917908f7872b14e/users/dd7668b9d5e14cc0954acdc7392a6fea/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.556794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 926/1847] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:42:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:45.576576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb8e43bfc36450aa917908f7872b14e, user_id=dd7668b9d5e14cc0954acdc7392a6fea, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:45.580373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:45.584707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:45.610742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.612224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:45.641356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0ebec-e1d3-488c-9e9e-608b29f9ad8f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:45.660529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:45.661439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:45.661722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:45.662004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] PATH_INFO: `/v3/projects/6884353dcb8044aab7ba3d007dfe177a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:45.663512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6884353dcb8044aab7ba3d007dfe177a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:45.665491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:45.710387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 922/1848] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:45 2022] PUT /identity/v3/projects/ebb8e43bfc36450aa917908f7872b14e/users/dd7668b9d5e14cc0954acdc7392a6fea/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:45.716059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.821304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6884353dcb8044aab7ba3d007dfe177a', 'request_id': 'req-4f979ac2-8e72-4a21-97d1-36a6c18774d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:45.886096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a620d80-611c-4726-b4e9-6054194d7c90 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:45.893916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6884353dcb8044aab7ba3d007dfe177a', 'request_id': 'req-4f979ac2-8e72-4a21-97d1-36a6c18774d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:45.905277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6884353dcb8044aab7ba3d007dfe177a', 'request_id': 'req-4f979ac2-8e72-4a21-97d1-36a6c18774d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:45.934370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] The token cache is being invalidate because project 6884353dcb8044aab7ba3d007dfe177a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:45.934934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:45.935686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:45.942639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 927/1849] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:45 2022] DELETE /identity/v3/projects/6884353dcb8044aab7ba3d007dfe177a => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:45.942639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:45.942639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.018778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:46.019722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.020124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.020508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.021442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:46.023355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:46.035089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 923/1850] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:45 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:46.039324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:46.041123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.054621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c727f-63ff-401a-a612-61baf197b8f6 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:46.062098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:46.063193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.063484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.073721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] PATH_INFO: `/v3/users/aec4b3131b9c4ebcbac1bc1931ee8934` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.074017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f979ac2-8e72-4a21-97d1-36a6c18774d1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:46.074092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aec4b3131b9c4ebcbac1bc1931ee8934)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:46.078060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:46.088499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:46.089212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.089496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.089774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.091694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.105404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:46.107581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:46.119375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 928/1851] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:46.128251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:46.133369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.148658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ffef9-ea2a-4b8b-b572-967fdd8eb18e service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:46.154109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f562ab-ffe0-4c31-a7b3-74eb0cba46ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:46.155624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f562ab-ffe0-4c31-a7b3-74eb0cba46ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.155624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f562ab-ffe0-4c31-a7b3-74eb0cba46ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.155959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f562ab-ffe0-4c31-a7b3-74eb0cba46ae admin admin] PATH_INFO: `/v3/projects/ebb8e43bfc36450aa917908f7872b14e/users/dd7668b9d5e14cc0954acdc7392a6fea/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.158204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f562ab-ffe0-4c31-a7b3-74eb0cba46ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb8e43bfc36450aa917908f7872b14e, user_id=dd7668b9d5e14cc0954acdc7392a6fea, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:46.161317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f562ab-ffe0-4c31-a7b3-74eb0cba46ae admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:46.178311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 929/1852] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:46 2022] PUT /identity/v3/projects/ebb8e43bfc36450aa917908f7872b14e/users/dd7668b9d5e14cc0954acdc7392a6fea/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:46.189250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.189578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.189849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.242781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] MFA Rules not processed for user `dd7668b9d5e14cc0954acdc7392a6fea`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:46.244094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.324429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 930/1853] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:46.404622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aec4b3131b9c4ebcbac1bc1931ee8934'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:46.405221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aec4b3131b9c4ebcbac1bc1931ee8934'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:46.442350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:46.443887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.522551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aec4b3131b9c4ebcbac1bc1931ee8934'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:46.531506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aec4b3131b9c4ebcbac1bc1931ee8934'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:46.560903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edf1c7-1fa9-4e4e-beba-9f2f267d13b0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:46.562771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aec4b3131b9c4ebcbac1bc1931ee8934'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:46.570311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:46.571985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.572277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.576276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.578633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.581873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 924/1854] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:46 2022] DELETE /identity/v3/users/aec4b3131b9c4ebcbac1bc1931ee8934 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:46.600415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:46.602343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:46.616626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 931/1855] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:46.866816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:46.868278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:46.941867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25d264-c8f7-4181-96af-0e1db056b898 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:46.961174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:46.961174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:46.961174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:46.961174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:46.961497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:47.045225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:47.045225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:47.054074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 925/1856] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:47.248962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:47.252433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:47.322462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d2fce9-d653-41bb-aabb-cecf49e3ef20 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:47.383236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:47.384193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:47.384478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:47.384756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] PATH_INFO: `/v3/users/ab5809ffcd2244c68e3f831594cef9d1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:47.386003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab5809ffcd2244c68e3f831594cef9d1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:47.387945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:47.422458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:47.424023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:47.455798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5180a190-6d7a-4829-b593-b6f60b027db9 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:47.455798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:47.456754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:47.457074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:47.457353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] PATH_INFO: `/v3/projects/9f86990085704d0f99f738518d38954d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:47.458530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f86990085704d0f99f738518d38954d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:47.460443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:47.557966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:47.576701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab5809ffcd2244c68e3f831594cef9d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.577117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab5809ffcd2244c68e3f831594cef9d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.598278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f86990085704d0f99f738518d38954d', 'request_id': 'req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.688516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab5809ffcd2244c68e3f831594cef9d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.710998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab5809ffcd2244c68e3f831594cef9d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.743103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-46a0960f-6705-4bf8-8e0f-46d12d9e3ad6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab5809ffcd2244c68e3f831594cef9d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.745091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f86990085704d0f99f738518d38954d', 'request_id': 'req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.758253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f86990085704d0f99f738518d38954d', 'request_id': 'req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.760447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 932/1857] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:47 2022] DELETE /identity/v3/users/ab5809ffcd2244c68e3f831594cef9d1 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:47.766105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:47.766733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:47.767248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:47.786015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] The token cache is being invalidate because project 9f86990085704d0f99f738518d38954d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:47.786342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.787000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e4937b20-6967-45af-9ee9-d2cf0b5e5f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:47.789954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 926/1858] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:47 2022] DELETE /identity/v3/projects/9f86990085704d0f99f738518d38954d => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:47.806587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66c17-1d57-4358-bb28-94da6e1d6c0f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:47.806923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66c17-1d57-4358-bb28-94da6e1d6c0f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:47.807223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded66c17-1d57-4358-bb28-94da6e1d6c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:47.818947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:47.827639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:47.846252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ded66c17-1d57-4358-bb28-94da6e1d6c0f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:47.849119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ded66c17-1d57-4358-bb28-94da6e1d6c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:47.944351 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:48.029499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 927/1859] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:48.030988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.031279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.031557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.038173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 933/1860] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:48.055590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.058264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:48.106294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:48.107733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:48.148401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 928/1861] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:48.167705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.190043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0c2d8-7f74-4e75-b682-1079a85c985c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.201828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.202623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.203040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.204261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.206251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:48.207491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.212017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.218911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f011008e-1024-4868-aeba-fb0e3bd1abbe None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.222954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.223791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.224158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.224778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.242108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.244046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.255977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 929/1862] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:48.267413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 934/1863] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:48.269927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.283612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121229c-ce6e-4ea1-9ce5-89e0b610e53d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.290508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.292380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.292972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.294151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.294939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.299652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.344535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.345968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:48.419598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb082ae5-8ba4-4811-84fc-97c08fbbbb73 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.424465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.425131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.426978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.426978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.427850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:48.484287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 930/1864] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:48 2022] POST /identity/v3/projects => generated 391 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:42:48.488441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.544577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.547119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.562991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 935/1865] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:48.593755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3f64a-ae6b-4eab-9343-910ae86418a2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.600289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.601414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.601885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.602383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.603334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.606427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.744988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 931/1866] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:48 2022] POST /identity/v3/users => generated 296 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:48.747741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.764361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aabe5a-f619-49aa-887f-1cf281fc0bd1 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.770260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.771901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.772025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.772235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.773046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.776607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.792763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 936/1867] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:48.793968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.805398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b34288-24cd-4110-b215-40ebfd6d63aa admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.811058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da35bd-3a5b-4324-8f2c-1b4489678101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.811984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da35bd-3a5b-4324-8f2c-1b4489678101 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.812465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da35bd-3a5b-4324-8f2c-1b4489678101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.812738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da35bd-3a5b-4324-8f2c-1b4489678101 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.813471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da35bd-3a5b-4324-8f2c-1b4489678101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.815127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da35bd-3a5b-4324-8f2c-1b4489678101 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.830022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 932/1868] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:48.833749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:48.859492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b486f-db5c-4093-b319-5d457c3f430e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:48.866206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:48.866862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.867223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.867586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] PATH_INFO: `/v3/projects/7aaa55a4668f4f21bf1d4106a4768967/users/366a14c9419a401bb6c8cedeeddcc5c7/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.887654 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aaa55a4668f4f21bf1d4106a4768967, user_id=366a14c9419a401bb6c8cedeeddcc5c7, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:48.891171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:48.973168 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:48.973458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:48.974023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:48.975063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 937/1869] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:48 2022] PUT /identity/v3/projects/7aaa55a4668f4f21bf1d4106a4768967/users/366a14c9419a401bb6c8cedeeddcc5c7/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:49.017775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] MFA Rules not processed for user `366a14c9419a401bb6c8cedeeddcc5c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:49.023077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:49.083798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 933/1870] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:49.088360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.152402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.153826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:49.154849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f30a4-5253-4fd7-98d1-a4facd950ed7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.162224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.167904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.167904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.167904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.167904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.168428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.175837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f942c79-eaa6-4a21-8cef-b79586d7babb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.183186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.188299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.188855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.189243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] PATH_INFO: `/v3/projects/70e65584be1f41dbbff58d9027d51213` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.190930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e65584be1f41dbbff58d9027d51213)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.197170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.262458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 938/1871] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:49 2022] POST /identity/v3/projects => generated 393 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:49.266518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.322422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e65584be1f41dbbff58d9027d51213', 'request_id': 'req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.336905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ec6c4d-b75e-4967-ade9-aaee3d8d799b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.385015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e65584be1f41dbbff58d9027d51213', 'request_id': 'req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.387639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:49.393216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e65584be1f41dbbff58d9027d51213', 'request_id': 'req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.396080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.396758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.397046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.397326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.398017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.399980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.411610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] The token cache is being invalidate because project 70e65584be1f41dbbff58d9027d51213 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:49.411930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.412674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.414738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 934/1872] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:49 2022] DELETE /identity/v3/projects/70e65584be1f41dbbff58d9027d51213 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:49.419060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.420513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:49.433713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd555bd-a9a7-4e3a-b82a-5b7efec0d251 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.438366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.439222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.439530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.439871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] PATH_INFO: `/v3/users/2fac8dc2d1f34a9b83e78e12ded548e6` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.441008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fac8dc2d1f34a9b83e78e12ded548e6)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.444556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.522404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 939/1873] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:49 2022] POST /identity/v3/users => generated 303 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:42:49.526603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.528243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:49.544681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e13de-7685-4184-8a78-d2fc4d958a98 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.551231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.553460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.553460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.553460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.554785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.558669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.570660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 940/1874] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:42:49.574786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.584520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c6ea4-b8bd-471b-a8e2-5e8ddc5be7da admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.589202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.590069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.590351 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.590627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] PATH_INFO: `/v3/projects/0901fb76ffde4a23a26e6dccba4bfd75/users/5c97b0ca66724cc2a472db286855456a/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.605881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0901fb76ffde4a23a26e6dccba4bfd75, user_id=5c97b0ca66724cc2a472db286855456a, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.608549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.697959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 941/1875] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:49 2022] PUT /identity/v3/projects/0901fb76ffde4a23a26e6dccba4bfd75/users/5c97b0ca66724cc2a472db286855456a/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:49.699202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.705059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fac8dc2d1f34a9b83e78e12ded548e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.706147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fac8dc2d1f34a9b83e78e12ded548e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.761929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fac8dc2d1f34a9b83e78e12ded548e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.771917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fac8dc2d1f34a9b83e78e12ded548e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.786260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fac8dc2d1f34a9b83e78e12ded548e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:49.796245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 935/1876] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:49 2022] DELETE /identity/v3/users/2fac8dc2d1f34a9b83e78e12ded548e6 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:49.801571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c9bbc-1998-424e-8f79-53ee7fe0b33c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.890907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74e011-5c8f-4ad2-a157-d15bede360b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.891850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74e011-5c8f-4ad2-a157-d15bede360b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.892316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74e011-5c8f-4ad2-a157-d15bede360b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.893902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74e011-5c8f-4ad2-a157-d15bede360b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.894889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74e011-5c8f-4ad2-a157-d15bede360b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.896811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74e011-5c8f-4ad2-a157-d15bede360b7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.912310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 942/1877] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:49 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:49.916146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:49.918309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:49.943973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb31b8-6451-4765-9bdf-5dbeef5e7d88 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:49.951711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:49.953032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.959077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.959077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] PATH_INFO: `/v3/projects/0901fb76ffde4a23a26e6dccba4bfd75/users/5c97b0ca66724cc2a472db286855456a/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:49.959077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0901fb76ffde4a23a26e6dccba4bfd75, user_id=5c97b0ca66724cc2a472db286855456a, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:49.960917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:49.973803 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 936/1878] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:49 2022] PUT /identity/v3/projects/0901fb76ffde4a23a26e6dccba4bfd75/users/5c97b0ca66724cc2a472db286855456a/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:49.987599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:49.987599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:49.987599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.013860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] MFA Rules not processed for user `5c97b0ca66724cc2a472db286855456a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:50.015074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:50.111357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 943/1879] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:50.122777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:50.124733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:50.206980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac679da-9fd1-4737-920d-acc2f61c0be1 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:50.224061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:50.231566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.232067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.232351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.247899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:50.249727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:50.279260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 937/1880] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:50.343506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:50.353008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f18b-2169-4cce-b2cc-fe79ac5a432b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:50.358818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:50.359489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.359817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.360136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.361925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:50.414655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:50.416065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:50.424716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:50.427052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f44f5-1c6f-4da8-ba2f-e26749ffd620 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:50.438346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 944/1881] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:50.487087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a1efce-e902-4c25-b673-ec9980d95551 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:50.493220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:50.496578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.496578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.496578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] PATH_INFO: `/v3/projects/7211b0a2321a4354bdabccbadc38c1f2` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.496578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7211b0a2321a4354bdabccbadc38c1f2)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:50.498609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:50.610205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7211b0a2321a4354bdabccbadc38c1f2', 'request_id': 'req-848012a9-ce0d-44f3-a0de-c8e0c34a5562'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:50.704571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7211b0a2321a4354bdabccbadc38c1f2', 'request_id': 'req-848012a9-ce0d-44f3-a0de-c8e0c34a5562'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:50.714186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7211b0a2321a4354bdabccbadc38c1f2', 'request_id': 'req-848012a9-ce0d-44f3-a0de-c8e0c34a5562'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:50.728319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] The token cache is being invalidate because project 7211b0a2321a4354bdabccbadc38c1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:50.728701 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:50.730001 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-848012a9-ce0d-44f3-a0de-c8e0c34a5562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:50.731753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 938/1882] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:50 2022] DELETE /identity/v3/projects/7211b0a2321a4354bdabccbadc38c1f2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:50.737022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.737317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.737596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.756287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:50.758001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:50.821623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 945/1883] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:50.829439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.829734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.830019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.852759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:50.855165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:50.866605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 939/1884] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:42:50.871071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:50.881356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c08ba-46ed-4285-b900-1e166b39072d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:50.886856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:50.887509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.887823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.888299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.889081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:50.889391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:50.890966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:50.903603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 946/1885] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:50.907565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:50.917887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864f480-73de-4ce2-aab8-c00293c5f772 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:50.922670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:50.923312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:50.923594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:50.923903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:50.924587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:50.926505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.008666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 940/1886] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:50 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:51.013323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.023419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727e2bc-44d2-477f-8f09-8594b6b4022e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.082904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.083996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.083996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.084375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.085154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.087238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.196356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 947/1887] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:51 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:51.207372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.212387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6628ac-71e2-4ae6-8df8-a8c3b1530beb admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.217837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.218413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.218702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.219551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.220410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.222571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.250116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 941/1888] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:51.256592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.272527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b5127-6b32-42f2-8d14-52973d94f815 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.277542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e424d2-fd0c-4d9d-9c96-34ac3935f724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.281146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e424d2-fd0c-4d9d-9c96-34ac3935f724 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.281146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e424d2-fd0c-4d9d-9c96-34ac3935f724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.281146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e424d2-fd0c-4d9d-9c96-34ac3935f724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.281146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e424d2-fd0c-4d9d-9c96-34ac3935f724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.281618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e424d2-fd0c-4d9d-9c96-34ac3935f724 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.292331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 948/1889] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:51.296577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.308358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec84c7cb-9056-4f32-a157-8aa1be8d3499 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.312951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.313997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.314272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.314670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] PATH_INFO: `/v3/projects/9fa000322b0e47ecbd1b7f9e01b508d1/users/9d1cc949807e49209db2f9c31b1543cf/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.332085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa000322b0e47ecbd1b7f9e01b508d1, user_id=9d1cc949807e49209db2f9c31b1543cf, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.338546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.419270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:51.425500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 942/1890] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:51 2022] PUT /identity/v3/projects/9fa000322b0e47ecbd1b7f9e01b508d1/users/9d1cc949807e49209db2f9c31b1543cf/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:51.439921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.440236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.440491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.466263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] MFA Rules not processed for user `9d1cc949807e49209db2f9c31b1543cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:51.467528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:51.531905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 949/1891] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:51.541434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.603554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e7bd2-dd7c-42d0-8eba-84c7b19544fd admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.608799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.609458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.609745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.610031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.610730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.612686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.743897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 943/1892] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:51 2022] POST /identity/v3/projects => generated 375 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:51.745753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.764081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695cb65-01e1-449e-b78f-11966b2b5bba None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.839127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.839127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.839127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.839127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.839748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.839748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.870141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:51.936886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 950/1893] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:51 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:42:51.938169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.949492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c5e592-2512-4be3-8171-dc4040246e3e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.982738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.982738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:51.982738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:51.982738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:51.983337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:51.983337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:51.983337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 944/1894] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:51.983568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:51.991938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b125ad41-5963-4e6c-bf14-93d759256920 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:51.997848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:51.998737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:52.027114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:52.027114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] PATH_INFO: `/v3/projects/0ec27faceae44070920e5f5cdeafbf9f/users/2e5b2c7e2c824058b9f2cdf996041c50/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:52.027114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec27faceae44070920e5f5cdeafbf9f, user_id=2e5b2c7e2c824058b9f2cdf996041c50, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:52.027114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:52.614426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 951/1895] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:51 2022] PUT /identity/v3/projects/0ec27faceae44070920e5f5cdeafbf9f/users/2e5b2c7e2c824058b9f2cdf996041c50/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:52.615285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:52.615285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c88886-e157-4d2c-bdfa-18231633a687 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:52.666540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b3e9a-c028-4e8a-97b5-3d305520c5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:52.667249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b3e9a-c028-4e8a-97b5-3d305520c5af admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:52.667534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b3e9a-c028-4e8a-97b5-3d305520c5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:52.667850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b3e9a-c028-4e8a-97b5-3d305520c5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:52.668578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b3e9a-c028-4e8a-97b5-3d305520c5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:52.670156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b3e9a-c028-4e8a-97b5-3d305520c5af admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:52.683969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 945/1896] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:42:52.685939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:52.696641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d2f4c8-ac24-47d7-93fd-e4836f98487c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:52.710428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:52.711352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:52.711640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:52.711950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] PATH_INFO: `/v3/projects/0ec27faceae44070920e5f5cdeafbf9f/users/2e5b2c7e2c824058b9f2cdf996041c50/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:52.714664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ec27faceae44070920e5f5cdeafbf9f, user_id=2e5b2c7e2c824058b9f2cdf996041c50, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:52.717506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:52.728183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 952/1897] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:42:52 2022] PUT /identity/v3/projects/0ec27faceae44070920e5f5cdeafbf9f/users/2e5b2c7e2c824058b9f2cdf996041c50/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:52.738626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:52.738918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:52.739194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:52.768555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] MFA Rules not processed for user `2e5b2c7e2c824058b9f2cdf996041c50`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:52.770080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:52.842498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 946/1898] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:52 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:53.446008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:53.448494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:53.519787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97033d70-fa12-4d19-967a-86755bd23cee admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:53.526176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae318d0-8347-4e20-9c19-4826a77d0beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:53.526981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae318d0-8347-4e20-9c19-4826a77d0beb service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:53.527321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae318d0-8347-4e20-9c19-4826a77d0beb service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:53.527608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae318d0-8347-4e20-9c19-4826a77d0beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:53.590295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae318d0-8347-4e20-9c19-4826a77d0beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:53.591779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae318d0-8347-4e20-9c19-4826a77d0beb service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:53.601601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 953/1899] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:54.914641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:54.922410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:55.020823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574d3329-b77e-420c-9a05-739a0271ea5b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:55.027420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:55.028328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:55.028615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:55.028898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] PATH_INFO: `/v3/users/5101278bd46a488cad55d1814d1b19ed` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:55.030109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5101278bd46a488cad55d1814d1b19ed)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:55.032118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:55.227920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5101278bd46a488cad55d1814d1b19ed'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:55.229562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5101278bd46a488cad55d1814d1b19ed'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:55.287260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5101278bd46a488cad55d1814d1b19ed'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:55.295753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5101278bd46a488cad55d1814d1b19ed'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:55.308980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5101278bd46a488cad55d1814d1b19ed'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:55.318014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:55.318607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 947/1900] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:54 2022] DELETE /identity/v3/users/5101278bd46a488cad55d1814d1b19ed => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:55.324107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:55.324551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:55.324933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:55.345683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:55.353413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:55.420184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 954/1901] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:55.426747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:55.428218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:55.485455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e5a2b-22c4-43e4-a00c-6fda6699ff97 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:55.488993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:55.489627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:55.489909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:55.490185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:55.500084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:55.501495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:55.511720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 948/1902] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:55.688588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:56.004385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:56.006017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:56.024143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1954a093-e141-49fe-8ea0-eae0c09d723d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:56.030400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:56.031820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:56.032416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:56.032888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] PATH_INFO: `/v3/projects/c681ba30e8d14965ba49b0a32656be07` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:56.034765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c681ba30e8d14965ba49b0a32656be07)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:56.036902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:56.173897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c681ba30e8d14965ba49b0a32656be07', 'request_id': 'req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.253340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c681ba30e8d14965ba49b0a32656be07', 'request_id': 'req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.260940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c681ba30e8d14965ba49b0a32656be07', 'request_id': 'req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.274549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] The token cache is being invalidate because project c681ba30e8d14965ba49b0a32656be07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:56.275082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.276077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.281683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 955/1903] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:56 2022] DELETE /identity/v3/projects/c681ba30e8d14965ba49b0a32656be07 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:56.285998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:56.298878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a49ba-9f7f-43fc-9e45-68cf28daacf3 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:56.371055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:56.372062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:56.372343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:56.372618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] PATH_INFO: `/v3/users/1f97b72334744d82b08bff082e9cf686` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:56.373793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f97b72334744d82b08bff082e9cf686)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:56.375737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:56.581338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f97b72334744d82b08bff082e9cf686'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.582672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f97b72334744d82b08bff082e9cf686'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.676607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f97b72334744d82b08bff082e9cf686'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.686834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f97b72334744d82b08bff082e9cf686'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.702159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f97b72334744d82b08bff082e9cf686'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:56.708563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 949/1904] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:56 2022] DELETE /identity/v3/users/1f97b72334744d82b08bff082e9cf686 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:56.860049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:56.971178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:56.972824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:56.987311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4ac4b3-eb9e-419c-9fe9-e1468b13fbf6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:57.053960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:57.054864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.055213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.055566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] PATH_INFO: `/v3/users/f58674cb6ccf46bba015872948016762` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.057008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58674cb6ccf46bba015872948016762)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:57.059312 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:57.181210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:57.183477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.198296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d31896-89d8-4198-aefd-5ebb7e76ae6e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:57.203378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:57.204310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.205378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.205664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] PATH_INFO: `/v3/projects/ab9df7e19d084a7885062171ddad1488` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.207426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9df7e19d084a7885062171ddad1488)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:57.210570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:57.277880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58674cb6ccf46bba015872948016762'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.278964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58674cb6ccf46bba015872948016762'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.348823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9df7e19d084a7885062171ddad1488', 'request_id': 'req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.350340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58674cb6ccf46bba015872948016762'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.365631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58674cb6ccf46bba015872948016762'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.373860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58674cb6ccf46bba015872948016762'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.380979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 956/1905] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:56 2022] DELETE /identity/v3/users/f58674cb6ccf46bba015872948016762 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:57.382266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:57.383807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.450290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dc3f5-35b6-4a67-a16b-499e916aaf3a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:57.452966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9df7e19d084a7885062171ddad1488', 'request_id': 'req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.458650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:57.459530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.464897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.464897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] PATH_INFO: `/v3/users/366a14c9419a401bb6c8cedeeddcc5c7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.465355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9df7e19d084a7885062171ddad1488', 'request_id': 'req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.465448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=366a14c9419a401bb6c8cedeeddcc5c7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:57.465448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:57.475400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] The token cache is being invalidate because project ab9df7e19d084a7885062171ddad1488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:57.475721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.476409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-56d8d71c-8cde-41d0-be7a-7e17cbb2e023 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.478321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 950/1906] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:57 2022] DELETE /identity/v3/projects/ab9df7e19d084a7885062171ddad1488 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:57.484355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.484355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.484355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.501170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:57.502378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.510089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 951/1907] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:57.510853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:57.521652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.546036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088ea89b-219f-46d0-a923-7aebb6d2442a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:57.551235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:57.552136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.552422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.552698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] PATH_INFO: `/v3/users/eca4ef6385604acabd71c907b3f7b43f` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.553890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca4ef6385604acabd71c907b3f7b43f)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:57.556030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:57.658113 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366a14c9419a401bb6c8cedeeddcc5c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.659342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366a14c9419a401bb6c8cedeeddcc5c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.779261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366a14c9419a401bb6c8cedeeddcc5c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.799868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366a14c9419a401bb6c8cedeeddcc5c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.807500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366a14c9419a401bb6c8cedeeddcc5c7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.815191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 957/1908] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:57 2022] DELETE /identity/v3/users/366a14c9419a401bb6c8cedeeddcc5c7 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:57.816363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:57.818986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca4ef6385604acabd71c907b3f7b43f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.819352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.819944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca4ef6385604acabd71c907b3f7b43f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.901987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc17fb-e94f-469a-a3dc-dc6b68ba66a5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:57.905709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:57.906342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.906627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.906903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.908781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.915208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca4ef6385604acabd71c907b3f7b43f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.922486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:57.927912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca4ef6385604acabd71c907b3f7b43f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.941205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-18e2754e-c422-4f52-ab31-fa2f8b0f7af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca4ef6385604acabd71c907b3f7b43f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:57.948171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 952/1909] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:57 2022] DELETE /identity/v3/users/eca4ef6385604acabd71c907b3f7b43f => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:57.949498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:57.949793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:57.950152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:57.987886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:57.989254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:57.995743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:57.998084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:58.012322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 958/1910] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:42:58.061604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 953/1911] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:58.071582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:58.073571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:58.084788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0877e707-b9b9-4e21-8e13-0813864896d4 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:58.143674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:58.144431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:58.144722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:58.145000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:58.155912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:58.157599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:58.169983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 959/1912] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:42:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:42:58.704641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:58.747966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:58.753821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:58.754065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:58.755537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:58.773061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f3f5a-7599-4c00-8bdd-0e840982c9d9 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:58.778301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f13a97-ab9e-43fc-87aa-de1c53253d62 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:58.782638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:58.783729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:58.784223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:58.784451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] PATH_INFO: `/v3/projects/48bec031e7774d6882a0ae5c51f89249` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:58.785951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] RBAC: Authorizing `identity:delete_project(project_id=48bec031e7774d6882a0ae5c51f89249)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:58.786053 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:58.787825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:58.788074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:58.788269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:58.788943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] PATH_INFO: `/v3/projects/aeb3e08cf36d4f5aad0fb6de90194bce` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:58.791708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeb3e08cf36d4f5aad0fb6de90194bce)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:58.795197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:58.916729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48bec031e7774d6882a0ae5c51f89249', 'request_id': 'req-e322fab8-ae93-4950-baab-61b51a776bad'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:58.958342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeb3e08cf36d4f5aad0fb6de90194bce', 'request_id': 'req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:58.984050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48bec031e7774d6882a0ae5c51f89249', 'request_id': 'req-e322fab8-ae93-4950-baab-61b51a776bad'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:58.990981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48bec031e7774d6882a0ae5c51f89249', 'request_id': 'req-e322fab8-ae93-4950-baab-61b51a776bad'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.004695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] The token cache is being invalidate because project 48bec031e7774d6882a0ae5c51f89249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:59.005218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.006132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.017602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 954/1913] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:58 2022] DELETE /identity/v3/projects/48bec031e7774d6882a0ae5c51f89249 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:59.017602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.017602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:59.037430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeb3e08cf36d4f5aad0fb6de90194bce', 'request_id': 'req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.050288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeb3e08cf36d4f5aad0fb6de90194bce', 'request_id': 'req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.067624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] The token cache is being invalidate because project aeb3e08cf36d4f5aad0fb6de90194bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:59.067978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.068640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b7b7e65-0eb5-4aaa-97df-3d743c7893c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.073140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 960/1914] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:58 2022] DELETE /identity/v3/projects/aeb3e08cf36d4f5aad0fb6de90194bce => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:42:59.073140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.073140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.073140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.095502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:59.097294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:59.104503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e322fab8-ae93-4950-baab-61b51a776bad admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:59.142201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:59.188131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 961/1915] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:59.188131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.188131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:59.193704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:59.194626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.194944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.195222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] PATH_INFO: `/v3/projects/7aaa55a4668f4f21bf1d4106a4768967` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.196329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3092bf6-6dcf-4e24-8ba2-200d37beea0c None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:59.196544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aaa55a4668f4f21bf1d4106a4768967)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:59.198509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:59.201280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:59.202142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.202433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.202690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] PATH_INFO: `/v3/users/dd7668b9d5e14cc0954acdc7392a6fea` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.204001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd7668b9d5e14cc0954acdc7392a6fea)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:59.205892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:59.217642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:42:59.371201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aaa55a4668f4f21bf1d4106a4768967', 'request_id': 'req-5f34f5de-e7a8-471f-984b-267a9879ef8e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.374025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd7668b9d5e14cc0954acdc7392a6fea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.374948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd7668b9d5e14cc0954acdc7392a6fea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.418170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aaa55a4668f4f21bf1d4106a4768967', 'request_id': 'req-5f34f5de-e7a8-471f-984b-267a9879ef8e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.424985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aaa55a4668f4f21bf1d4106a4768967', 'request_id': 'req-5f34f5de-e7a8-471f-984b-267a9879ef8e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.438472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] The token cache is being invalidate because project 7aaa55a4668f4f21bf1d4106a4768967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:42:59.438758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.439428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5f34f5de-e7a8-471f-984b-267a9879ef8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.441020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 955/1916] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:59 2022] DELETE /identity/v3/projects/7aaa55a4668f4f21bf1d4106a4768967 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:59.444993 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.445288 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.445568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.460962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd7668b9d5e14cc0954acdc7392a6fea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.466206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:42:59.467501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:59.483689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd7668b9d5e14cc0954acdc7392a6fea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.502007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd7668b9d5e14cc0954acdc7392a6fea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.510709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 962/1917] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:59 2022] DELETE /identity/v3/users/dd7668b9d5e14cc0954acdc7392a6fea => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:59.511709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.513189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:42:59.547597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 956/1918] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:42:59.550371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.569751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7887de4-9711-4141-b32d-021232f48f66 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:59.623520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112bd6-5927-45df-a72a-6a7eb2a9bcd0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:59.628628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:59.629520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.629804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.630084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] PATH_INFO: `/v3/users/5c97b0ca66724cc2a472db286855456a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.631309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c97b0ca66724cc2a472db286855456a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:59.633430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:59.651988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:59.652882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.653223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.653581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.654408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:42:59.654768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:59.657021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:59.670328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 957/1919] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:42:59.674418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.685737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e9d58-666d-444d-8fb5-596a1cb12b1b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:59.692979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:59.694118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.694631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.695140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.696511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:59.704015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:59.829961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 958/1920] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:42:59 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:42:59.830963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.867322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c97b0ca66724cc2a472db286855456a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.868611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c97b0ca66724cc2a472db286855456a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.903908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5abb95-ba23-4dc2-8a1e-4724f5aed7ec admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:42:59.908761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:42:59.909439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:42:59.909728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:42:59.910010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:42:59.910700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:42:59.912665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:42:59.945250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c97b0ca66724cc2a472db286855456a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.962724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c97b0ca66724cc2a472db286855456a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.970550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c97b0ca66724cc2a472db286855456a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:42:59.981667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 963/1921] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:42:59 2022] DELETE /identity/v3/users/5c97b0ca66724cc2a472db286855456a => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:42:59.995410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:42:59.997876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.043445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 959/1922] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:42:59 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:00.052785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.055362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.072226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e47db-e3a5-46aa-a717-1f8a21758c72 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.091028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65b8114-39c0-497d-95e1-8ad0b629c402 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.096807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.097469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.097749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.098023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] PATH_INFO: `/v3/projects/ebb8e43bfc36450aa917908f7872b14e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.099340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb8e43bfc36450aa917908f7872b14e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.103264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.149298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.150128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.150264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.150595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.151347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.153709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.168969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 960/1923] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:00.170499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.184193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73aecf-70d7-4b40-93fb-c8907324dc8c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.189623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.190460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.190848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.191227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.192159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.193971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.208119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 961/1924] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:00.212305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.223765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1f797-b875-4966-b041-10c991ffa727 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.243194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb8e43bfc36450aa917908f7872b14e', 'request_id': 'req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.315570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb8e43bfc36450aa917908f7872b14e', 'request_id': 'req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.325007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb8e43bfc36450aa917908f7872b14e', 'request_id': 'req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.326619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60ba7b-4e95-437a-9276-f6ef45c0da38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.327559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60ba7b-4e95-437a-9276-f6ef45c0da38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.327874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60ba7b-4e95-437a-9276-f6ef45c0da38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.328153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60ba7b-4e95-437a-9276-f6ef45c0da38 admin admin] PATH_INFO: `/v3/projects/cfc2ebc096c741f3af08bfb7f0c1805f/users/ab2dc8d098b54e7987c89dbc44908995/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.339825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:00.346690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] The token cache is being invalidate because project ebb8e43bfc36450aa917908f7872b14e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:00.350228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.350228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1222ecc7-7d76-4030-b6bd-d6b2c6e87f3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.350228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 964/1925] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:42:59 2022] DELETE /identity/v3/projects/ebb8e43bfc36450aa917908f7872b14e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:00.352022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60ba7b-4e95-437a-9276-f6ef45c0da38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfc2ebc096c741f3af08bfb7f0c1805f, user_id=ab2dc8d098b54e7987c89dbc44908995, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.354791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e599959-fff3-4940-bdd7-d5aa10159e08 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.355159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60ba7b-4e95-437a-9276-f6ef45c0da38 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.355439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e599959-fff3-4940-bdd7-d5aa10159e08 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.355816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e599959-fff3-4940-bdd7-d5aa10159e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.377980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5e599959-fff3-4940-bdd7-d5aa10159e08 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:00.379245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5e599959-fff3-4940-bdd7-d5aa10159e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.397138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 965/1926] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:00.410201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.410755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.411132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.420714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:00.430858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:00.433612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.446740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 966/1927] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:00.448791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.455577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 962/1928] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:00 2022] PUT /identity/v3/projects/cfc2ebc096c741f3af08bfb7f0c1805f/users/ab2dc8d098b54e7987c89dbc44908995/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:00.495592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.495592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.495592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.495892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b34434-03fb-4cab-b60b-5f0a60ef29a3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.557896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] MFA Rules not processed for user `ab2dc8d098b54e7987c89dbc44908995`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:00.559155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.626690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.627480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.627899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.628348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.629164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:00.629649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.631288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.647808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 967/1929] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:00.651988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.653084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.669292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae129a9c-75d6-451d-acbd-7ae1339209e2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.674540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.675541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.675894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.676182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] PATH_INFO: `/v3/projects/0901fb76ffde4a23a26e6dccba4bfd75` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.677541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0901fb76ffde4a23a26e6dccba4bfd75)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.685359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.685537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 963/1930] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:00.687261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.689479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.779894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c6ef8-d6d3-4f45-935f-7225940ccce6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:00.780726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:00.781697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:00.782661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:00.782661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:00.785344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.821766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0901fb76ffde4a23a26e6dccba4bfd75', 'request_id': 'req-023d4773-5329-4c28-9c6d-f512f1b19e52'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.864918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0901fb76ffde4a23a26e6dccba4bfd75', 'request_id': 'req-023d4773-5329-4c28-9c6d-f512f1b19e52'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.876772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0901fb76ffde4a23a26e6dccba4bfd75', 'request_id': 'req-023d4773-5329-4c28-9c6d-f512f1b19e52'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.895777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] The token cache is being invalidate because project 0901fb76ffde4a23a26e6dccba4bfd75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:00.896184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.897184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:00.900171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 968/1931] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:00 2022] DELETE /identity/v3/projects/0901fb76ffde4a23a26e6dccba4bfd75 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:00.901082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.904692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.910055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:00.911599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:00.922942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 964/1932] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:00.924821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:00.928212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:00.958977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228c84d-88a8-42a7-bd14-e28bc1b9e535 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.019750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d4773-5329-4c28-9c6d-f512f1b19e52 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.028126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b893a02-7d51-4c7f-b0ef-9077b265615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.028895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893a02-7d51-4c7f-b0ef-9077b265615f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.029229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893a02-7d51-4c7f-b0ef-9077b265615f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.029546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b893a02-7d51-4c7f-b0ef-9077b265615f admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.030359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b893a02-7d51-4c7f-b0ef-9077b265615f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.032643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b893a02-7d51-4c7f-b0ef-9077b265615f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.072268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.073247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.073464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.073629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.074281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.076313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.126278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 969/1933] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:00 2022] POST /identity/v3/projects => generated 379 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:01.126278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.126278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.126278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.142008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 965/1934] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:00 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:01.149825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.150032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:01.150032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:01.161542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:01.238025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1624b3-dff3-4c60-9062-97f494c3fdcf admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.238688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 970/1935] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:01.246879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb70569-389f-4aba-80d2-b7923627b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.246879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb70569-389f-4aba-80d2-b7923627b8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.246879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb70569-389f-4aba-80d2-b7923627b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.246879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb70569-389f-4aba-80d2-b7923627b8b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.247267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb70569-389f-4aba-80d2-b7923627b8b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.247507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.248860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb70569-389f-4aba-80d2-b7923627b8b9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.254834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:01.279864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5882e-9941-48e2-9028-6ea166515cb4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.289594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.290888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.292207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.292743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.294756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.297951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.348200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 966/1936] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:01 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:01.348200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.348200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.348984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.365052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 971/1937] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:01 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:01.366146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.376392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:01.378099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:01.380634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df89cad-f22c-4ce6-8e8c-699281bf6bf5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.387112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.388074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.388523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.388943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.393645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 967/1938] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:01.393645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.402683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.405484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.408158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e430480-1f07-4d34-88eb-ac36de0766a3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.414914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.416144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.416636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.417231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.418310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.420741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.426247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 972/1939] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:01.427319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.436483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 968/1940] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:01.437702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.446146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe60b23f-cae6-43b8-91eb-705baa58a2a6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.451753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8861b8c-6759-4cc3-a9cf-36c95ff4494e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.455938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.457255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.457623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.458121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.458909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.459490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:01.460019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.460135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.460640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.461138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.462817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.465869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.475954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.497267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 969/1941] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:01.503594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.503594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:01.503819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 973/1942] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:01.512316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.523912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868910d0-6b21-4ada-b92a-3b8deab5e659 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.530206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.531244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.531603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.531986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] PATH_INFO: `/v3/projects/2dac3cf8db1b4d12a232f76938daa34b/users/3282924f701543879422dffe0106abd4/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.535597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd33be-8871-413e-b53a-d0c790e9c222 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.561836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.562973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.563323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.563718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] PATH_INFO: `/v3/projects/2a7b049e41e647e59b152bce6ec0df9e/users/8dfce76fc75f4ee290db30afe95e2e33/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.586376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dac3cf8db1b4d12a232f76938daa34b, user_id=3282924f701543879422dffe0106abd4, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.593812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.597744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7b049e41e647e59b152bce6ec0df9e, user_id=8dfce76fc75f4ee290db30afe95e2e33, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.603754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.730473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 970/1943] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:01 2022] PUT /identity/v3/projects/2a7b049e41e647e59b152bce6ec0df9e/users/8dfce76fc75f4ee290db30afe95e2e33/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:01.733674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.741164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 974/1944] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:01 2022] PUT /identity/v3/projects/2dac3cf8db1b4d12a232f76938daa34b/users/3282924f701543879422dffe0106abd4/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:01.742461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:01.842591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc3241-1633-46bb-880f-883a7e336fb1 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.855513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10ab0d-b8f6-4b6b-9a26-5a11677e1d53 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:01.860899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421cf5b3-5bff-4d6f-a786-6c4e12a68776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.861596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cf5b3-5bff-4d6f-a786-6c4e12a68776 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.861883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cf5b3-5bff-4d6f-a786-6c4e12a68776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.862357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421cf5b3-5bff-4d6f-a786-6c4e12a68776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.863168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cf5b3-5bff-4d6f-a786-6c4e12a68776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.865050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421cf5b3-5bff-4d6f-a786-6c4e12a68776 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:01.877332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 975/1945] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:01.879617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.881781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.882382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.933905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] MFA Rules not processed for user `3282924f701543879422dffe0106abd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:01.935548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:01.940216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81caa0e-39eb-4d9b-b8ae-53bf40dfafb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:01.940901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81caa0e-39eb-4d9b-b8ae-53bf40dfafb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:01.941187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81caa0e-39eb-4d9b-b8ae-53bf40dfafb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:01.945153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81caa0e-39eb-4d9b-b8ae-53bf40dfafb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:01.945963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81caa0e-39eb-4d9b-b8ae-53bf40dfafb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:01.954809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81caa0e-39eb-4d9b-b8ae-53bf40dfafb7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.022880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 976/1946] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:02.024299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.041846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d68ec-8725-4302-bc70-2c84610758cb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.049082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.050013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.050296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.050571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] PATH_INFO: `/v3/projects/2a7b049e41e647e59b152bce6ec0df9e/users/8dfce76fc75f4ee290db30afe95e2e33/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.052684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a7b049e41e647e59b152bce6ec0df9e, user_id=8dfce76fc75f4ee290db30afe95e2e33, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.055382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.068086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 977/1947] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:02 2022] PUT /identity/v3/projects/2a7b049e41e647e59b152bce6ec0df9e/users/8dfce76fc75f4ee290db30afe95e2e33/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:02.068923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.089203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:02.107907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 971/1948] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:01 2022] POST /identity/v3/projects => generated 409 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:02.110239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.110811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.111250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.159007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] MFA Rules not processed for user `8dfce76fc75f4ee290db30afe95e2e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:02.160912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:02.189145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2c1115-6eba-46f2-9e67-648bfb563403 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.295566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.295936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.296443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.296980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.300272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.304450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.308449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 972/1949] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:02.309269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.330454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0bbba-b5c8-4f92-be15-bc4bed8541ad None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.337636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.339028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.339467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.339877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.340601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.342788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.426571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:02.433850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 973/1950] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:02 2022] POST /identity/v3/users => generated 305 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:02.435144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.437841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:02.440388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 978/1951] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:02 2022] POST /identity/v3/projects => generated 381 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:02.441216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.518611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9cc8d5-bf89-49f7-a721-64543829d8e1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.526821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.528109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.528672 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.529205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.530599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.533658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.542132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eb07ce-0c51-46bd-9528-597901fb31fc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.553514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 979/1952] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:02.553954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.571822 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3027e727-1d2e-4d17-a5f8-ca58e4198ca1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.580632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.582570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.583178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.583794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.585182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.589780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.644488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.645642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.646739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.647247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.659746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 980/1953] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:02 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:02.665167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.682874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65567f1-beb7-41bc-99b4-639cd1b86c07 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.692251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.693717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.694560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.695259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.696883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.705286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.723436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 981/1954] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:02.727444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.744533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.746341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.747392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39211970-ce77-4282-b4c6-6b8e03f34cf0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.758029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.759291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 974/1955] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:02.759899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.760457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.761167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.761857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.763748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.767991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.774123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5688d83-6f5f-48c3-95b3-f9b5a554b431 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.780304 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1772cc-bea9-47c8-8c97-786b43cea900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.781921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1772cc-bea9-47c8-8c97-786b43cea900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.782929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1772cc-bea9-47c8-8c97-786b43cea900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.783419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1772cc-bea9-47c8-8c97-786b43cea900 admin admin] PATH_INFO: `/v3/projects/2e996005fbc245398126ec419a56c995/users/e5b41ed1e31148e08494dc69b76b594e/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.800493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:02.809631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 982/1956] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:02.816209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1772cc-bea9-47c8-8c97-786b43cea900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e996005fbc245398126ec419a56c995, user_id=e5b41ed1e31148e08494dc69b76b594e, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.818475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:02.818653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1772cc-bea9-47c8-8c97-786b43cea900 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.843271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416de355-b006-4cee-8ae5-2b9b083c8885 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:02.852939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:02.855099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.855344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.855827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] PATH_INFO: `/v3/projects/a7f275e9bb1247bc975ea49e918de106/users/89042b8963be47f8b0dc83e304009a47/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.893537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f275e9bb1247bc975ea49e918de106, user_id=89042b8963be47f8b0dc83e304009a47, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:02.894600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:02.901724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 975/1957] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:02 2022] PUT /identity/v3/projects/2e996005fbc245398126ec419a56c995/users/e5b41ed1e31148e08494dc69b76b594e/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:02.914956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:02.915333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:02.915655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:02.946203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] MFA Rules not processed for user `e5b41ed1e31148e08494dc69b76b594e`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:02.948196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:02.969634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 983/1958] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:02 2022] PUT /identity/v3/projects/a7f275e9bb1247bc975ea49e918de106/users/89042b8963be47f8b0dc83e304009a47/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:02.972969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:02.974336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.035301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 976/1959] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:02 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:03.039349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.048965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab356db-ef2b-43c9-8e52-d678794ac9f7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.060618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.061508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.061941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.062410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.063151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.064900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.085973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 984/1960] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:03.089250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.102301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec2a57-8ff2-41b1-b076-b569f44b4d75 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.109470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d57b1-bab4-42dd-ad70-fc837adc6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.110742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d57b1-bab4-42dd-ad70-fc837adc6679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.111035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d57b1-bab4-42dd-ad70-fc837adc6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.111315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d57b1-bab4-42dd-ad70-fc837adc6679 admin admin] PATH_INFO: `/v3/projects/a7f275e9bb1247bc975ea49e918de106/users/89042b8963be47f8b0dc83e304009a47/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.113769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d57b1-bab4-42dd-ad70-fc837adc6679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f275e9bb1247bc975ea49e918de106, user_id=89042b8963be47f8b0dc83e304009a47, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.117212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d57b1-bab4-42dd-ad70-fc837adc6679 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.127070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7295a61c-162c-45b7-9afe-ce8d23902126 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.129909 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 985/1961] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:03 2022] PUT /identity/v3/projects/a7f275e9bb1247bc975ea49e918de106/users/89042b8963be47f8b0dc83e304009a47/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:03.141164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.141438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.141714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.176798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] MFA Rules not processed for user `89042b8963be47f8b0dc83e304009a47`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:03.178175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:03.192464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.193219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.193575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.193858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.194563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.196569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.271355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 986/1962] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:03 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:03.278768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.284789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:03.313278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 977/1963] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:03 2022] POST /identity/v3/projects => generated 407 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:03.316065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.338280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3426fad-7508-4b05-a13a-625ed251d7bd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.386992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853298a-3e11-4d05-b203-da236f601bf0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.391936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.392908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.393197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.393500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.395647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.396591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.397099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.397904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.398644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.401023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.468538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.470441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.481549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 987/1964] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:03.485117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 978/1965] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:03 2022] POST /identity/v3/users => generated 310 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:03.486311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.489995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:03.500217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11aca07-6a0f-4f9e-bdb3-4bf504eca663 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.505222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.505915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.507331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.507331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.507501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.509175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.520255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 988/1966] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:03.524447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.534742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675631c-e61d-413c-b748-8d418c972c0a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.540273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.541339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.541622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.541901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] PATH_INFO: `/v3/projects/43e7d94c306d42dab9b3d3d2f0866a85/users/75bf66a5069a4d5d82f932e06028d674/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.557491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e7d94c306d42dab9b3d3d2f0866a85, user_id=75bf66a5069a4d5d82f932e06028d674, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.560329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.596319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:03.654041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 979/1967] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:03 2022] PUT /identity/v3/projects/43e7d94c306d42dab9b3d3d2f0866a85/users/75bf66a5069a4d5d82f932e06028d674/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:03.658972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.671390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0bec8-5ef5-4e24-bbc0-5a32ac4ee822 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.754819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc4c29-5f1d-4ed2-a5ba-01089a9c2ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.754819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4c29-5f1d-4ed2-a5ba-01089a9c2ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.754819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4c29-5f1d-4ed2-a5ba-01089a9c2ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.754819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc4c29-5f1d-4ed2-a5ba-01089a9c2ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.755288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4c29-5f1d-4ed2-a5ba-01089a9c2ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.755288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc4c29-5f1d-4ed2-a5ba-01089a9c2ad5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.769278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 989/1968] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:03.771956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:03.782770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857c0db-663d-44d8-8530-390cdf8ce0a6 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:03.791512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:03.792466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.792749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.793020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] PATH_INFO: `/v3/projects/43e7d94c306d42dab9b3d3d2f0866a85/users/75bf66a5069a4d5d82f932e06028d674/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.795046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e7d94c306d42dab9b3d3d2f0866a85, user_id=75bf66a5069a4d5d82f932e06028d674, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:03.798430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:03.813829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 980/1969] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:03 2022] PUT /identity/v3/projects/43e7d94c306d42dab9b3d3d2f0866a85/users/75bf66a5069a4d5d82f932e06028d674/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:03.825365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:03.825677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:03.825901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:03.867002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] MFA Rules not processed for user `75bf66a5069a4d5d82f932e06028d674`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:03.869150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:03.972026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 990/1970] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:03 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:04.004782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:04.071271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:04.096202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd837b8-f246-4ad0-a1df-560fa1bc04e3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:04.102143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:04.106646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:04.107438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:04.107758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:04.108116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:04.120329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:04.122250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:04.134346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 981/1971] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:04 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:04.419930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:04.433318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:04.499328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b11b-952e-457c-a72b-08fb94a1e408 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:04.502677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:04.507872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:04.508978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:04.509207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:04.509491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] PATH_INFO: `/v3/users/ab2dc8d098b54e7987c89dbc44908995` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:04.511556 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2dc8d098b54e7987c89dbc44908995)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:04.513691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:04.520487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8e635-4b33-46cd-b5f6-abd41eead99d service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:04.527138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:04.528594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:04.529044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:04.529703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] PATH_INFO: `/v3/users/3282924f701543879422dffe0106abd4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:04.531287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3282924f701543879422dffe0106abd4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:04.534019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:04.536721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:04.734115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2dc8d098b54e7987c89dbc44908995'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.734784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2dc8d098b54e7987c89dbc44908995'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.757827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3282924f701543879422dffe0106abd4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.758470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3282924f701543879422dffe0106abd4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.789525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2dc8d098b54e7987c89dbc44908995'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.801387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2dc8d098b54e7987c89dbc44908995'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.808800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2dc8d098b54e7987c89dbc44908995'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.814779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 991/1972] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:04 2022] DELETE /identity/v3/users/ab2dc8d098b54e7987c89dbc44908995 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:04.815562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:04.816954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:04.846557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3282924f701543879422dffe0106abd4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.853748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3282924f701543879422dffe0106abd4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.859554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:04.865853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b509d327-a237-4a0b-a81b-9b6c5ba48697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3282924f701543879422dffe0106abd4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:04.872476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 982/1973] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:04 2022] DELETE /identity/v3/users/3282924f701543879422dffe0106abd4 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:04.875251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb6306c-6d5c-4953-bf1d-52e2b907eebe admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:04.876583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d99ec4-4a7e-4b9b-b291-feb166076481 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:04.876875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d99ec4-4a7e-4b9b-b291-feb166076481 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:04.877155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d99ec4-4a7e-4b9b-b291-feb166076481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:04.894467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-82d99ec4-4a7e-4b9b-b291-feb166076481 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:04.896089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-82d99ec4-4a7e-4b9b-b291-feb166076481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:04.918462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:04.925257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:04.927528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:04.927528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:04.927528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:04.928798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:04.963084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 983/1974] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:04.964528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:04.964815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:04.967969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:04.992640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:04.994536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:04.995255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:04.996484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:05.003561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 992/1975] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:04 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:05.004473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:05.008463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 984/1976] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:05.008953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:05.017528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:05.019514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:05.030332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:05.076316 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:05.096580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a8d7c-d87b-4f17-89bc-b20cda40f48d service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:05.102646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:05.104173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:05.104707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:05.104909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37136642-706d-492e-acc4-e2b4caa84e1e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:05.105275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:05.110211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:05.111358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:05.111358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:05.111540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:05.123858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:05.124207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:05.126197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:05.126868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:05.139337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 985/1977] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:05.139837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 993/1978] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:05.178872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:05.338891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:05.779166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:05.781843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:05.800242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37099bb1-da87-4f6a-b9b2-66d51b84c43d service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:05.805603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:05.806698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:05.806868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:05.807146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] PATH_INFO: `/v3/projects/2dac3cf8db1b4d12a232f76938daa34b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:05.808982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dac3cf8db1b4d12a232f76938daa34b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:05.813277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:05.854854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:05.947966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:05.949728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:05.965628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dac3cf8db1b4d12a232f76938daa34b', 'request_id': 'req-61261aec-53fc-420b-bd03-5cdd353d2b39'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:05.968383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042464b7-88a9-4ef5-a169-0855646a6b9c service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:06.058171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dac3cf8db1b4d12a232f76938daa34b', 'request_id': 'req-61261aec-53fc-420b-bd03-5cdd353d2b39'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.058981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:06.060182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:06.060687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:06.061178 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] PATH_INFO: `/v3/projects/cfc2ebc096c741f3af08bfb7f0c1805f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:06.063041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfc2ebc096c741f3af08bfb7f0c1805f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:06.065036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dac3cf8db1b4d12a232f76938daa34b', 'request_id': 'req-61261aec-53fc-420b-bd03-5cdd353d2b39'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.066124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:06.086177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] The token cache is being invalidate because project 2dac3cf8db1b4d12a232f76938daa34b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:06.086488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.087139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.090111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 986/1979] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:05 2022] DELETE /identity/v3/projects/2dac3cf8db1b4d12a232f76938daa34b => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:06.095308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:06.110914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:06.125793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61261aec-53fc-420b-bd03-5cdd353d2b39 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:06.131089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:06.132001 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:06.132302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:06.132614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] PATH_INFO: `/v3/users/89042b8963be47f8b0dc83e304009a47` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:06.134011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=89042b8963be47f8b0dc83e304009a47)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:06.136533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:06.205917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfc2ebc096c741f3af08bfb7f0c1805f', 'request_id': 'req-7d20f8ce-2236-426c-8f75-595ec67d5bce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.269881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfc2ebc096c741f3af08bfb7f0c1805f', 'request_id': 'req-7d20f8ce-2236-426c-8f75-595ec67d5bce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.277779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfc2ebc096c741f3af08bfb7f0c1805f', 'request_id': 'req-7d20f8ce-2236-426c-8f75-595ec67d5bce'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.294570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] The token cache is being invalidate because project cfc2ebc096c741f3af08bfb7f0c1805f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:06.295183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.296452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.303724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 994/1980] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:05 2022] DELETE /identity/v3/projects/cfc2ebc096c741f3af08bfb7f0c1805f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:06.319420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89042b8963be47f8b0dc83e304009a47'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.320594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:06.320725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89042b8963be47f8b0dc83e304009a47'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.323530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:06.414342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20f8ce-2236-426c-8f75-595ec67d5bce admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:06.417454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:06.418246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89042b8963be47f8b0dc83e304009a47'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.422465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:06.423652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:06.424060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:06.424439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] PATH_INFO: `/v3/users/8dfce76fc75f4ee290db30afe95e2e33` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:06.425791 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfce76fc75f4ee290db30afe95e2e33)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:06.429776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:06.430259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89042b8963be47f8b0dc83e304009a47'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.469127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89042b8963be47f8b0dc83e304009a47'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.474590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:06.487272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 987/1981] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:06 2022] DELETE /identity/v3/users/89042b8963be47f8b0dc83e304009a47 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:06.758570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfce76fc75f4ee290db30afe95e2e33'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.759981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfce76fc75f4ee290db30afe95e2e33'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.875345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfce76fc75f4ee290db30afe95e2e33'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.888692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfce76fc75f4ee290db30afe95e2e33'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.896670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfce76fc75f4ee290db30afe95e2e33'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:06.904380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 995/1982] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:06 2022] DELETE /identity/v3/users/8dfce76fc75f4ee290db30afe95e2e33 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:07.274080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:07.274911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:07.304197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:07.379352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9bd3-f7b6-4f07-87bb-03809a3b97ff admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:07.399237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:07.402383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:07.459315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:07.473031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:07.473907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:07.474189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:07.474643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] PATH_INFO: `/v3/projects/a7f275e9bb1247bc975ea49e918de106` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:07.476307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f275e9bb1247bc975ea49e918de106)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:07.481650 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:07.575262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1418e6-0030-4749-83dd-7f55ba4cd368 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:07.586615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:07.586615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:07.586615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:07.587057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] PATH_INFO: `/v3/projects/2a7b049e41e647e59b152bce6ec0df9e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:07.592580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a7b049e41e647e59b152bce6ec0df9e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:07.608675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f275e9bb1247bc975ea49e918de106', 'request_id': 'req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.611128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:07.738743 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f275e9bb1247bc975ea49e918de106', 'request_id': 'req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.748874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f275e9bb1247bc975ea49e918de106', 'request_id': 'req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.772739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] The token cache is being invalidate because project a7f275e9bb1247bc975ea49e918de106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:07.773445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.774579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8f17f115-ad1a-4416-9d59-ed72d4bfe16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.781125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 988/1983] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:07 2022] DELETE /identity/v3/projects/a7f275e9bb1247bc975ea49e918de106 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:07.793148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82b944-a9cf-4e68-83fd-592a0d5092a2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:07.793872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82b944-a9cf-4e68-83fd-592a0d5092a2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:07.794597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a82b944-a9cf-4e68-83fd-592a0d5092a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:07.817298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a7b049e41e647e59b152bce6ec0df9e', 'request_id': 'req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.847224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-9a82b944-a9cf-4e68-83fd-592a0d5092a2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:07.850775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9a82b944-a9cf-4e68-83fd-592a0d5092a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:07.905532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a7b049e41e647e59b152bce6ec0df9e', 'request_id': 'req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.926691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a7b049e41e647e59b152bce6ec0df9e', 'request_id': 'req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.940322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] The token cache is being invalidate because project 2a7b049e41e647e59b152bce6ec0df9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:07.943799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.943799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1f48949e-dd40-425f-b2b2-5dfeb4e8b4f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:07.945644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 996/1984] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:07 2022] DELETE /identity/v3/projects/2a7b049e41e647e59b152bce6ec0df9e => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:07.949474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdf08e-f87c-4cab-8f5d-65993b504d11 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:07.952333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdf08e-f87c-4cab-8f5d-65993b504d11 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:07.952775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdf08e-f87c-4cab-8f5d-65993b504d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:07.983467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0fcdf08e-f87c-4cab-8f5d-65993b504d11 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:07.987838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 989/1985] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:07.988047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0fcdf08e-f87c-4cab-8f5d-65993b504d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:07.992493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:07.992892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:07.993188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.020490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 997/1986] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:08.034553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:08.037491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:08.038116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.038628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.039764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.066632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 990/1987] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:08.071054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.083237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:08.083237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:08.086087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885b252-ee2f-4e1a-ac7a-efe1e950c8ef None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.096258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 998/1988] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:08.098544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.100725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:08.100832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.100832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.100832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.103089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:08.104546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.105894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.106162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.117888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55c3ab-c81a-469b-8426-ddbc275bad60 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.123597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.124330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.124547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 991/1989] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:08.124655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.124870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.125940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:08.126249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.128160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.129174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.141079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326d0465-1e4b-4366-8b0c-2bd4b6b1c8bf admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.141520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 999/1990] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:08.147686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.154171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.157318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.157825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.159019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.163559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.167603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.168185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ede6d4d-6649-4df4-bbb2-4d51ba2bf8c6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.176710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.177273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.177562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.178358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.178704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.180708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.249611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:08.260307 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 992/1991] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:08.262158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1000/1992] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:08.263616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.270161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.387033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c8578-f5cb-4c8d-bf6c-7f06dbdc2c50 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.389022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef35ed52-d592-441b-9b85-591da5f9315d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.394112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.394976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.395194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.398390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.398390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.398619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.398619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.398619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.398619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.399081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.401987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.404727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.511300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 993/1993] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:08.516525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.529555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1001/1994] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:08.534281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.538237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943adecb-6f94-439f-bb0b-77c3b84b7223 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.545283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5dac1-bfbc-43ac-9c93-a705743bd4e5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.546953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.549556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.549556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.549556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.550554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.552969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.553607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.553857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.554249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.554624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.555459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.557759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.569435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 994/1995] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:08.570025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1002/1996] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:08.576602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.585031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.592305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379e95c3-2fac-475e-ba07-94e900256845 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.596953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aacf40-183f-4efa-baa4-47d8ef4a9a4f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.598644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.599822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.600411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.601084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.602327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.602521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.603344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.603888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.604236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.605058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.605735 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.606771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.621945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1003/1997] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:08.628247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.638283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 995/1998] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:08.643179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.655793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325c04f3-3e1a-4099-8631-2299461e96e7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.657329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590a9ba-7967-4064-adbb-f2c97fa21924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.658751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590a9ba-7967-4064-adbb-f2c97fa21924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.659265 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590a9ba-7967-4064-adbb-f2c97fa21924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.659824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590a9ba-7967-4064-adbb-f2c97fa21924 admin admin] PATH_INFO: `/v3/projects/427330caa288445c81143ff1e3b48e49/users/d4a4491f663943839557f27e86187918/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.669497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec4b13-892b-4385-b10c-9488af84a945 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:08.680637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21490c9-96ee-4893-a8b2-0578c4417a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:08.682432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21490c9-96ee-4893-a8b2-0578c4417a89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.682847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21490c9-96ee-4893-a8b2-0578c4417a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.683256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21490c9-96ee-4893-a8b2-0578c4417a89 admin admin] PATH_INFO: `/v3/projects/8518347e59ff4feea367cca79c4ac0e0/users/1e9b7945b254480ebaa521ac09118522/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.691054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590a9ba-7967-4064-adbb-f2c97fa21924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=427330caa288445c81143ff1e3b48e49, user_id=d4a4491f663943839557f27e86187918, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.694097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590a9ba-7967-4064-adbb-f2c97fa21924 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.711953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21490c9-96ee-4893-a8b2-0578c4417a89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8518347e59ff4feea367cca79c4ac0e0, user_id=1e9b7945b254480ebaa521ac09118522, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:08.711953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21490c9-96ee-4893-a8b2-0578c4417a89 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:08.791143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1004/1999] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:08 2022] PUT /identity/v3/projects/427330caa288445c81143ff1e3b48e49/users/d4a4491f663943839557f27e86187918/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:08.803572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.804040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.804437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.809861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:08.818301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 996/2000] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:08 2022] PUT /identity/v3/projects/8518347e59ff4feea367cca79c4ac0e0/users/1e9b7945b254480ebaa521ac09118522/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:08.835181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:08.835766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:08.836330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:08.855100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] MFA Rules not processed for user `d4a4491f663943839557f27e86187918`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:08.857259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:08.880340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] MFA Rules not processed for user `1e9b7945b254480ebaa521ac09118522`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:08.882341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:08.916791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:08.922296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:08.963806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1005/2001] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:08.967345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:08.998216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 997/2002] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:09.003725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.054682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.059700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf4f1-1b56-4e10-8b38-65cf1d594205 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.065545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.066452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.067401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.067807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.068637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.072211 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.111021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.157902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf971f-a16f-4a6a-b180-337300a9c41a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.167023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.168025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.168406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.168898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.170295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.174165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.198938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.207706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1006/2003] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:08 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:09.212495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.286248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb382c8-e0b7-4f6f-8e0f-de6499182209 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.293069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.294519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.294519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.294519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.295322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.295700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 998/2004] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:09 2022] POST /identity/v3/projects => generated 385 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:09.300023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.301115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.377247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1007/2005] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:09 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:09.377454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.378950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.381585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf958e2-d8c3-44e9-acc4-55cbf04e27db admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.386872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.387801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.388234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.388691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.389654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.392284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.397854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19b7ee-603d-4516-bd85-8cc798bdeb32 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.406750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.407986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.408465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.408949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.410373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.414008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.428624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1008/2006] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:09.432774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.447037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d3a30-f04d-46ff-9570-3171cc173307 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.455834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.457827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.458415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.459009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] PATH_INFO: `/v3/projects/a734219adbd843618e9bb27059e2ba91/users/8328e0c1ae9e4ca4b91fad85370de57f/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.459543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 999/2007] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:09 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:09.464858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.477963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f389fa-d8ac-4ed7-89e1-23e627baa98f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.484758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.485819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.486195 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.486855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.487432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.490116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.493112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a734219adbd843618e9bb27059e2ba91, user_id=8328e0c1ae9e4ca4b91fad85370de57f, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.497122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.506077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1000/2008] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:09.506308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.507881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.522863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab99549-3827-473e-bbaa-9c009b32157c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.529323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.530241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.531914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.531914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.531914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.533607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.547451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1001/2009] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:09.552031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.564444 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be452512-d9ef-4ef2-97f2-bbc00b4efb8d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.570976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.573032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.573484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.573955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] PATH_INFO: `/v3/projects/86d772f38d0544228631df1df53973bb/users/50df0e2448cb45aaac0d269f4e76a9d3/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.577023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1009/2010] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:09 2022] PUT /identity/v3/projects/a734219adbd843618e9bb27059e2ba91/users/8328e0c1ae9e4ca4b91fad85370de57f/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:09.582279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.599344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d772f38d0544228631df1df53973bb, user_id=50df0e2448cb45aaac0d269f4e76a9d3, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.604662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.660072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2c229-ecee-46eb-b312-46a088d12111 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.666891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feb81c6-67fa-43f0-9e8a-cb2843b445b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.667745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb81c6-67fa-43f0-9e8a-cb2843b445b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.668305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb81c6-67fa-43f0-9e8a-cb2843b445b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.668699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feb81c6-67fa-43f0-9e8a-cb2843b445b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.669461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb81c6-67fa-43f0-9e8a-cb2843b445b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.671195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feb81c6-67fa-43f0-9e8a-cb2843b445b3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.677136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1002/2011] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:09 2022] PUT /identity/v3/projects/86d772f38d0544228631df1df53973bb/users/50df0e2448cb45aaac0d269f4e76a9d3/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:09.682659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.689730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1010/2012] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:09.690674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.691223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.691518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.691899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.728193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] MFA Rules not processed for user `50df0e2448cb45aaac0d269f4e76a9d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:09.735253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:09.819259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3491880-042a-4963-888d-ca9293512c1b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.825013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1011/2013] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:09.826329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.828047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:09.828813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145fad5-cf03-4afd-af3c-60478cb0a411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.829731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145fad5-cf03-4afd-af3c-60478cb0a411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.830010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145fad5-cf03-4afd-af3c-60478cb0a411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.833262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145fad5-cf03-4afd-af3c-60478cb0a411 admin admin] PATH_INFO: `/v3/projects/a734219adbd843618e9bb27059e2ba91/users/8328e0c1ae9e4ca4b91fad85370de57f/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.833262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145fad5-cf03-4afd-af3c-60478cb0a411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a734219adbd843618e9bb27059e2ba91, user_id=8328e0c1ae9e4ca4b91fad85370de57f, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.835533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145fad5-cf03-4afd-af3c-60478cb0a411 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.842075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68125cab-335e-47ff-b829-b5e271383b5e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:09.848848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1003/2014] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:09 2022] PUT /identity/v3/projects/a734219adbd843618e9bb27059e2ba91/users/8328e0c1ae9e4ca4b91fad85370de57f/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:09.858930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:09.860720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378589-075b-4550-993d-eed26b02687b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.861408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378589-075b-4550-993d-eed26b02687b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.861757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8378589-075b-4550-993d-eed26b02687b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.915892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f8378589-075b-4550-993d-eed26b02687b None None] MFA Rules not processed for user `8328e0c1ae9e4ca4b91fad85370de57f`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:09.917861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f8378589-075b-4550-993d-eed26b02687b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:09.925748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:09.926432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:09.926715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:09.926989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:09.927873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:09.929835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:09.978882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:10.021521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1004/2015] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:10.034892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8378589-075b-4550-993d-eed26b02687b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.036430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f8378589-075b-4550-993d-eed26b02687b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:10.068694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1012/2016] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:09 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:10.074133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.099253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8378589-075b-4550-993d-eed26b02687b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.162507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70743195-a9c6-4224-9e5d-3ed033380c03 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.167952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:10.170879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.171560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.171881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.172156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.172942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.175024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.176377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.177228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.177577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.177918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.247728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.249255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.260175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1005/2017] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:10.301452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1013/2018] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:10 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:10.304946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.321632 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb016f7-6b49-47dc-9b71-15d5a7a37983 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.333370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.334827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.334967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.335495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.336849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.340634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.352954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1006/2019] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:10.357481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.367763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a52d4-46e3-45bc-bc74-4ac73c6d4b36 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.373185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.374083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.374365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.374640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] PATH_INFO: `/v3/projects/6d37eacb82e04bb9b3cf67f696af6529/users/60e38277669b4ce8b367fa29bd20ff3a/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.394436 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d37eacb82e04bb9b3cf67f696af6529, user_id=60e38277669b4ce8b367fa29bd20ff3a, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.397423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.408055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.426015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827854-6420-41d0-bbde-b54c945348b9 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.429911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.430634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.431016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.431400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.472356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1014/2020] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:10 2022] PUT /identity/v3/projects/6d37eacb82e04bb9b3cf67f696af6529/users/60e38277669b4ce8b367fa29bd20ff3a/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:10.477517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.496541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.497999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.508381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1007/2021] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:10.529079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:10.550287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0e39db-e67f-496f-a747-0046d677972c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.555445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f827f40-a8f5-47ba-8758-abf90662d64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.558055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f827f40-a8f5-47ba-8758-abf90662d64c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.558055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f827f40-a8f5-47ba-8758-abf90662d64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.558055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f827f40-a8f5-47ba-8758-abf90662d64c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.558771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f827f40-a8f5-47ba-8758-abf90662d64c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.560511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f827f40-a8f5-47ba-8758-abf90662d64c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.573671 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1015/2022] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:10 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:10.577801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.589303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71367495-20f4-4f6d-8ae0-411c28f057e5 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.594331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.595214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.595506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.595823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] PATH_INFO: `/v3/projects/6d37eacb82e04bb9b3cf67f696af6529/users/60e38277669b4ce8b367fa29bd20ff3a/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.597852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d37eacb82e04bb9b3cf67f696af6529, user_id=60e38277669b4ce8b367fa29bd20ff3a, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.600549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.611404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1008/2023] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:10 2022] PUT /identity/v3/projects/6d37eacb82e04bb9b3cf67f696af6529/users/60e38277669b4ce8b367fa29bd20ff3a/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:10.622680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.623187 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.623618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.657042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] MFA Rules not processed for user `60e38277669b4ce8b367fa29bd20ff3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:10.658274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:10.765312 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1016/2024] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:10.780938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:10.782947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:10.849194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1a058f-568c-4200-b658-70a45747a66a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:10.853403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d011b-9e1e-432c-a673-e141b8d93027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:10.854204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d011b-9e1e-432c-a673-e141b8d93027 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:10.854389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d011b-9e1e-432c-a673-e141b8d93027 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:10.854663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d011b-9e1e-432c-a673-e141b8d93027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:10.878164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:10.926671 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d011b-9e1e-432c-a673-e141b8d93027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:10.928521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d011b-9e1e-432c-a673-e141b8d93027 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:10.948913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1009/2025] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:11.188960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:11.287094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf78d2-62ab-4c43-8b6d-a400d6451811 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:11.292085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:11.293086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:11.293373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:11.293648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] PATH_INFO: `/v3/users/d4a4491f663943839557f27e86187918` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:11.296438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a4491f663943839557f27e86187918)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:11.301406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:11.605449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a4491f663943839557f27e86187918'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:11.611280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a4491f663943839557f27e86187918'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:11.676300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a4491f663943839557f27e86187918'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:11.687898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:11.690292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a4491f663943839557f27e86187918'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:11.698166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a4491f663943839557f27e86187918'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:11.704459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1017/2026] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:11 2022] DELETE /identity/v3/users/d4a4491f663943839557f27e86187918 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:11.710266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:11.710968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:11.710968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:11.730265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:11.731565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:11.798406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1010/2027] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:11.808863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:11.810303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:11.873886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58256314-dccf-4901-a06e-d3afc85c1ce6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:11.876391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:11.877044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:11.877327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:11.877600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:11.886854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:11.888518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:11.898360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1018/2028] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:12.337074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:12.339804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:12.353007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98bccd8-6d37-497f-94f7-cc138448f105 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:12.358402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:12.359267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:12.359567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:12.359880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] PATH_INFO: `/v3/projects/427330caa288445c81143ff1e3b48e49` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:12.361250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] RBAC: Authorizing `identity:delete_project(project_id=427330caa288445c81143ff1e3b48e49)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:12.363181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:12.511421 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '427330caa288445c81143ff1e3b48e49', 'request_id': 'req-f20b6ba7-877c-49ba-aa05-f6af93ededea'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.595429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '427330caa288445c81143ff1e3b48e49', 'request_id': 'req-f20b6ba7-877c-49ba-aa05-f6af93ededea'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.601742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '427330caa288445c81143ff1e3b48e49', 'request_id': 'req-f20b6ba7-877c-49ba-aa05-f6af93ededea'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.613693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] The token cache is being invalidate because project 427330caa288445c81143ff1e3b48e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:12.613978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.614642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.616209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1011/2029] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:12 2022] DELETE /identity/v3/projects/427330caa288445c81143ff1e3b48e49 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:12.620264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:12.629906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa7438-318c-43b7-80f0-f6f64ffbfb68 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:12.705986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:12.707276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:12.707691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:12.708191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] PATH_INFO: `/v3/users/8328e0c1ae9e4ca4b91fad85370de57f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:12.709671 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8328e0c1ae9e4ca4b91fad85370de57f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:12.711806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:12.922990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8328e0c1ae9e4ca4b91fad85370de57f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.923589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8328e0c1ae9e4ca4b91fad85370de57f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:12.989216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8328e0c1ae9e4ca4b91fad85370de57f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.001944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8328e0c1ae9e4ca4b91fad85370de57f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.008590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e33c0cf0-ed47-4018-b0be-b1e7f51037f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8328e0c1ae9e4ca4b91fad85370de57f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.015160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1019/2030] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:12 2022] DELETE /identity/v3/users/8328e0c1ae9e4ca4b91fad85370de57f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:13.390927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:13.392762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:13.406810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b6ba7-877c-49ba-aa05-f6af93ededea admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:13.452319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:13.464657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:13.465515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:13.465797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:13.466072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] PATH_INFO: `/v3/projects/a734219adbd843618e9bb27059e2ba91` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:13.467246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a734219adbd843618e9bb27059e2ba91)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:13.469209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:13.584457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a734219adbd843618e9bb27059e2ba91', 'request_id': 'req-7598a506-5cf4-4160-8e63-ab597d492853'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.663357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a734219adbd843618e9bb27059e2ba91', 'request_id': 'req-7598a506-5cf4-4160-8e63-ab597d492853'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.669569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a734219adbd843618e9bb27059e2ba91', 'request_id': 'req-7598a506-5cf4-4160-8e63-ab597d492853'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.680623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] The token cache is being invalidate because project a734219adbd843618e9bb27059e2ba91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:13.680910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.681536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7598a506-5cf4-4160-8e63-ab597d492853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:13.684700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1012/2031] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:13 2022] DELETE /identity/v3/projects/a734219adbd843618e9bb27059e2ba91 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:13.688787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:13.689225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:13.689346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:13.701506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:13.707431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:13.708681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:13.766730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1020/2032] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:13.774637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:13.774925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:13.775201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:13.796181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:13.799477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:13.805739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1013/2033] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:13.809935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:13.819910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e109e58e-20f7-43d6-a8c4-0b416c89c766 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:13.825307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:13.826217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:13.826703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:13.826703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:13.827344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:13.828165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:13.829838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:13.842771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1021/2034] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:13.848367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:13.858611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7acfce-9541-49a4-a0a4-1d6a1be515a8 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:13.863955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:13.864662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:13.864952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:13.867234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:13.867234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:13.868025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:13.954301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1014/2035] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:13 2022] POST /identity/v3/projects => generated 375 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:13.958611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:13.970201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5d21b-1a2f-4965-919a-a8c1de22ac77 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.034805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.035471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.035860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.036139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.036825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.041853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.150533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1022/2036] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:13 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:14.155178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.165332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c72f058-bb52-4569-b6b9-58b75ec1d5d7 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.169860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.170554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.170884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.171113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.171820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.173417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.183829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1015/2037] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:14.187972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.199158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca1ba01-308c-4c51-bee1-9f7191501ed1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.203855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6838576-3969-43bd-93d0-b866326c484f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.204513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6838576-3969-43bd-93d0-b866326c484f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.204801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6838576-3969-43bd-93d0-b866326c484f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.205076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6838576-3969-43bd-93d0-b866326c484f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.205796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6838576-3969-43bd-93d0-b866326c484f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.207383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6838576-3969-43bd-93d0-b866326c484f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.218937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1023/2038] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:14.223283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.233417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ffe1b-842d-4ad2-b51d-1ba3a876da60 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.249575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.250651 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.250957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.251236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] PATH_INFO: `/v3/projects/4641e9973f43434f884a4a510f7f94b5/users/30acf5bcefa94de88c63b7745adddaf2/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.269889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4641e9973f43434f884a4a510f7f94b5, user_id=30acf5bcefa94de88c63b7745adddaf2, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.272630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.350245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1016/2039] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:14 2022] PUT /identity/v3/projects/4641e9973f43434f884a4a510f7f94b5/users/30acf5bcefa94de88c63b7745adddaf2/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:14.361722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.362010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.362284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.391932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] MFA Rules not processed for user `30acf5bcefa94de88c63b7745adddaf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:14.393415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:14.485076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.485694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1024/2040] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:14.602829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1b33c-163a-4200-bfe2-d6e09a7c2e86 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.608952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.610267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.610569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.610859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.611875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.614692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.717907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:14.725280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1017/2041] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:14 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:14.730650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.736688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:14.742816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbbaf0e-2ed2-408e-a86c-142dbb3085a0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.836514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.837747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.837747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.838008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.839566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.843038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.914509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1025/2042] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:14 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:14.915836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.927738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18a8e0-8434-44d8-abec-65cb5169a395 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.933445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.933445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.933916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.934293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:14.935124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:14.940836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:14.959954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1018/2043] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:14.964620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:14.977266 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f52289-b24c-4b35-a324-7e0be1f7df50 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:14.982269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:14.983296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:14.983656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:14.984060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] PATH_INFO: `/v3/projects/f6baa00401c24d3b9b7ee61d3e736ff8/users/48682f2a7d684e17830e695ea836fa93/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:15.022815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6baa00401c24d3b9b7ee61d3e736ff8, user_id=48682f2a7d684e17830e695ea836fa93, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:15.026072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:15.118149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1026/2044] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:14 2022] PUT /identity/v3/projects/f6baa00401c24d3b9b7ee61d3e736ff8/users/48682f2a7d684e17830e695ea836fa93/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:15.122505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:15.136210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a9875a-5691-4f86-be4d-de5cfa2354cf admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:15.264343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:15.266072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:15.272940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:15.274746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:15.275160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:15.276207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:15.277054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:15.278846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:15.292352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1019/2045] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:15.294577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:15.305505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87757e5e-7874-47a8-8adc-8588c8184d4d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:15.317780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba2117a-ced7-415d-bb8f-2c12f6edbc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:15.317780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2117a-ced7-415d-bb8f-2c12f6edbc98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:15.317780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2117a-ced7-415d-bb8f-2c12f6edbc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:15.317780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba2117a-ced7-415d-bb8f-2c12f6edbc98 admin admin] PATH_INFO: `/v3/projects/f6baa00401c24d3b9b7ee61d3e736ff8/users/48682f2a7d684e17830e695ea836fa93/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:15.334156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2117a-ced7-415d-bb8f-2c12f6edbc98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6baa00401c24d3b9b7ee61d3e736ff8, user_id=48682f2a7d684e17830e695ea836fa93, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:15.337132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba2117a-ced7-415d-bb8f-2c12f6edbc98 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:15.364449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1020/2046] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:15 2022] PUT /identity/v3/projects/f6baa00401c24d3b9b7ee61d3e736ff8/users/48682f2a7d684e17830e695ea836fa93/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:15.379337 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872be84-dea9-43f3-b109-44fcfd06ca4e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:15.390221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:15.390221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:15.391693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:15.457016 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] MFA Rules not processed for user `48682f2a7d684e17830e695ea836fa93`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:15.458875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:15.523572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:15.524258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:15.524649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:15.525054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:15.528152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:15.554091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1021/2047] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:15.614098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:15.646276 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:15.647880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:15.667972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1027/2048] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:15.875432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:15.876878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:15.888396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63c855-66da-4a65-9a42-7d617bf4b61b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:15.892267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:15.893313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:15.893600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:15.893879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:15.947976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:15.951555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:15.953032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:15.961917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1022/2049] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:17.580023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:17.591902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b414d16-91ad-4563-85ec-88c6f35fd85f service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:17.605480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:17.605480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:17.605480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:17.605480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:17.607386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:17.680798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:17.682447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:17.694900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1028/2050] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:19.047214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:19.050004 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:19.141217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699e3cc-e81e-478d-af8b-213c9a6e77ae service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:19.148019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:19.148019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.148019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.148019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] PATH_INFO: `/v3/users/e5b41ed1e31148e08494dc69b76b594e` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.150246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5b41ed1e31148e08494dc69b76b594e)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:19.150913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:19.356414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:19.366778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46572105-ebd6-419d-b094-760a2f9de13d service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:19.371346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:19.372239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.372513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.372784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] PATH_INFO: `/v3/users/30acf5bcefa94de88c63b7745adddaf2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.373981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30acf5bcefa94de88c63b7745adddaf2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:19.375950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:19.487690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5b41ed1e31148e08494dc69b76b594e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.488423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5b41ed1e31148e08494dc69b76b594e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.641093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5b41ed1e31148e08494dc69b76b594e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.651765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5b41ed1e31148e08494dc69b76b594e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.671054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bf57af4c-95b6-4aa7-b363-dd1fc082e473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5b41ed1e31148e08494dc69b76b594e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.676778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30acf5bcefa94de88c63b7745adddaf2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.677502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30acf5bcefa94de88c63b7745adddaf2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.682212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1023/2051] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:19 2022] DELETE /identity/v3/users/e5b41ed1e31148e08494dc69b76b594e => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:19.686353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.688569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.688569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.706597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:19.707950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:19.771517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1024/2052] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:19.782977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30acf5bcefa94de88c63b7745adddaf2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.788802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:19.790302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:19.800455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30acf5bcefa94de88c63b7745adddaf2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.806277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:19.808573 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1af499f5-d954-49f2-a63c-b6336da13d07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30acf5bcefa94de88c63b7745adddaf2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:19.818836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1029/2053] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:19 2022] DELETE /identity/v3/users/30acf5bcefa94de88c63b7745adddaf2 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:19.820409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.820699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.821035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.842704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:19.843956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:19.854670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7e8e-c12e-42c5-807f-c8e302ec2c25 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:19.895689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:19.919860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1030/2054] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:19.921059 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:19.922312 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:19.922601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:19.923260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.923543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.923883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.935049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2cee2c-07c4-48ab-ba60-7df791b4e65d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:19.937714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:19.939284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:19.940968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:19.941835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.943159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.943500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] PATH_INFO: `/v3/users/9583c43bffb24329b5e06b1e749affd8` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.945272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9583c43bffb24329b5e06b1e749affd8)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:19.947462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:19.958582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1025/2055] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:19.958582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:19.966480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa0576d-5926-4501-9875-7805257068c0 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:19.970523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:19.971169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:19.971455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:19.971764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:19.982578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:19.984204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:19.996972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1026/2056] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:20.238025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9583c43bffb24329b5e06b1e749affd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.238741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9583c43bffb24329b5e06b1e749affd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.337664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9583c43bffb24329b5e06b1e749affd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.351800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9583c43bffb24329b5e06b1e749affd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.362620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9583c43bffb24329b5e06b1e749affd8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.370452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1031/2057] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:19 2022] DELETE /identity/v3/users/9583c43bffb24329b5e06b1e749affd8 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:20.512233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:20.514302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:20.532058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2225c7-5064-4c5f-bcd7-ab7fd16d2f26 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:20.591341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:20.595746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:20.595899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:20.596737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:20.597187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:20.597460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] PATH_INFO: `/v3/projects/2e996005fbc245398126ec419a56c995` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:20.598714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e996005fbc245398126ec419a56c995)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:20.601989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:20.610831 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d8103-87de-4de9-a3c5-8e5ad5cbcbf2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:20.616128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:20.625977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:20.626143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:20.626425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] PATH_INFO: `/v3/projects/4641e9973f43434f884a4a510f7f94b5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:20.628694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] RBAC: Authorizing `identity:delete_project(project_id=4641e9973f43434f884a4a510f7f94b5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:20.634237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:20.758010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4641e9973f43434f884a4a510f7f94b5', 'request_id': 'req-5b0f412e-7cf4-4853-96b2-1606459ecfff'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.764351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e996005fbc245398126ec419a56c995', 'request_id': 'req-0ddd3c37-f55e-47a0-be1a-f393daa58294'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.846483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4641e9973f43434f884a4a510f7f94b5', 'request_id': 'req-5b0f412e-7cf4-4853-96b2-1606459ecfff'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.855006 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4641e9973f43434f884a4a510f7f94b5', 'request_id': 'req-5b0f412e-7cf4-4853-96b2-1606459ecfff'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.870297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] The token cache is being invalidate because project 4641e9973f43434f884a4a510f7f94b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:20.870681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.871775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.873634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1032/2058] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:20 2022] DELETE /identity/v3/projects/4641e9973f43434f884a4a510f7f94b5 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:20.874880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:20.876507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:20.921789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e996005fbc245398126ec419a56c995', 'request_id': 'req-0ddd3c37-f55e-47a0-be1a-f393daa58294'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.928614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:20.931531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:20.935061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e996005fbc245398126ec419a56c995', 'request_id': 'req-0ddd3c37-f55e-47a0-be1a-f393daa58294'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.953039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] The token cache is being invalidate because project 2e996005fbc245398126ec419a56c995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:20.953425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.954019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:20.955551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1027/2059] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:20 2022] DELETE /identity/v3/projects/2e996005fbc245398126ec419a56c995 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:20.956492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:20.957622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f412e-7cf4-4853-96b2-1606459ecfff admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:20.957787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:20.964893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:20.965837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:20.966121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:20.966545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] PATH_INFO: `/v3/projects/7baa5449b82446198b16e944162aa85b` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:20.967770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7baa5449b82446198b16e944162aa85b)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:20.969693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:21.042998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd3c37-f55e-47a0-be1a-f393daa58294 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:21.048330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:21.051006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.051440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.051757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] PATH_INFO: `/v3/users/48682f2a7d684e17830e695ea836fa93` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.053645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] RBAC: Authorizing `identity:delete_user(user_id=48682f2a7d684e17830e695ea836fa93)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:21.055572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:21.118053 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7baa5449b82446198b16e944162aa85b', 'request_id': 'req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.208111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7baa5449b82446198b16e944162aa85b', 'request_id': 'req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.216650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7baa5449b82446198b16e944162aa85b', 'request_id': 'req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.234861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] The token cache is being invalidate because project 7baa5449b82446198b16e944162aa85b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:21.235350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.236474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.246937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1033/2060] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:20 2022] DELETE /identity/v3/projects/7baa5449b82446198b16e944162aa85b => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:21.246937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:21.246937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:21.274902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48682f2a7d684e17830e695ea836fa93'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.276263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48682f2a7d684e17830e695ea836fa93'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.356272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20d5a3a-7146-4cf1-9eb6-9a850a7e25d8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:21.362449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:21.363560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.364107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.364198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] PATH_INFO: `/v3/users/75bf66a5069a4d5d82f932e06028d674` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.365405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bf66a5069a4d5d82f932e06028d674)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:21.367349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:21.390203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48682f2a7d684e17830e695ea836fa93'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.401546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48682f2a7d684e17830e695ea836fa93'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.424658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b3dcf003-a6d1-4325-8227-43b30cce934f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48682f2a7d684e17830e695ea836fa93'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.431516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1028/2061] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:20 2022] DELETE /identity/v3/users/48682f2a7d684e17830e695ea836fa93 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:21.441694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ea9ed-b56e-4117-bf6b-44ec4a57a82b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.442094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ea9ed-b56e-4117-bf6b-44ec4a57a82b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.442465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ea9ed-b56e-4117-bf6b-44ec4a57a82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.471085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-393ea9ed-b56e-4117-bf6b-44ec4a57a82b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:21.472353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-393ea9ed-b56e-4117-bf6b-44ec4a57a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:21.564705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bf66a5069a4d5d82f932e06028d674'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.565682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bf66a5069a4d5d82f932e06028d674'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.575289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1029/2062] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:21.586741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.587027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.587306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.607533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:21.608702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:21.617144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1030/2063] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:21.622997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:21.628756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:21.634538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b8ef7-f5dd-4e0c-908d-851d24ea222f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:21.639575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:21.640311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.640593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.640866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.641607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:21.641911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:21.643862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:21.653000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bf66a5069a4d5d82f932e06028d674'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.659444 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1031/2064] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:43:21.665284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:21.672225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bf66a5069a4d5d82f932e06028d674'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.673898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:21.681678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04214bd-5ba2-424f-ae77-1466c60f3927 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:21.683975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bf66a5069a4d5d82f932e06028d674'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:21.688426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:21.689928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.690364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.690785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.692509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:21.693884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:21.696374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1034/2065] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:21 2022] DELETE /identity/v3/users/75bf66a5069a4d5d82f932e06028d674 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:21.781383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1032/2066] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:21 2022] POST /identity/v3/projects => generated 355 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:21.782904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:21.862756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb5b67-3362-47c8-9439-f11c0f3f14cd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:21.922296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:21.930921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:21.930921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:21.930921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:21.930921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:21.930921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:21.983719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:21.983719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.002954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18a12f-cc7d-4021-859a-aed8fc8c30c4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.007941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.011793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.011793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.011793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] PATH_INFO: `/v3/projects/f6baa00401c24d3b9b7ee61d3e736ff8` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.011793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6baa00401c24d3b9b7ee61d3e736ff8)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.013863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.038969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1035/2067] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:21 2022] POST /identity/v3/users => generated 278 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:22.040237 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.050818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9574fee-87bf-4999-bbd5-9e9a29046800 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.055540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.056269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.056541 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.056804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.058041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.060629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.070994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1036/2068] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:22.077637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.094002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12631f-68dd-4782-af7e-9fc5422a007d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.098728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.099369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.099681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.102631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.102631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.102631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.117407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1037/2069] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:22.118458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.129669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aa6c6-f849-43c2-88c2-d365368baaea admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.142584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6baa00401c24d3b9b7ee61d3e736ff8', 'request_id': 'req-22a12b15-3832-4725-9420-63e514d9bc21'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.199293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24a6eb-7dc6-4a45-8a5f-e3ad9c169583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.199293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24a6eb-7dc6-4a45-8a5f-e3ad9c169583 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.199293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24a6eb-7dc6-4a45-8a5f-e3ad9c169583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.199293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24a6eb-7dc6-4a45-8a5f-e3ad9c169583 admin admin] PATH_INFO: `/v3/projects/1e0ae1f2cb9245a4a0ae5f9c97260480/users/fa6f09a3984c496293fb95c80edda583/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.209840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24a6eb-7dc6-4a45-8a5f-e3ad9c169583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e0ae1f2cb9245a4a0ae5f9c97260480, user_id=fa6f09a3984c496293fb95c80edda583, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.212639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24a6eb-7dc6-4a45-8a5f-e3ad9c169583 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.223777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6baa00401c24d3b9b7ee61d3e736ff8', 'request_id': 'req-22a12b15-3832-4725-9420-63e514d9bc21'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.231102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6baa00401c24d3b9b7ee61d3e736ff8', 'request_id': 'req-22a12b15-3832-4725-9420-63e514d9bc21'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.242817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] The token cache is being invalidate because project f6baa00401c24d3b9b7ee61d3e736ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:22.243109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.243855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.245782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1033/2070] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:21 2022] DELETE /identity/v3/projects/f6baa00401c24d3b9b7ee61d3e736ff8 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:22.246665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.247977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.261385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a12b15-3832-4725-9420-63e514d9bc21 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.266308 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.267167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.267907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.268188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] PATH_INFO: `/v3/projects/43e7d94c306d42dab9b3d3d2f0866a85` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.269439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43e7d94c306d42dab9b3d3d2f0866a85)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.271360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.316291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1038/2071] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:22 2022] PUT /identity/v3/projects/1e0ae1f2cb9245a4a0ae5f9c97260480/users/fa6f09a3984c496293fb95c80edda583/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:22.317969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0720641-5e3d-4796-80af-f95bd01706ea None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.318368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0720641-5e3d-4796-80af-f95bd01706ea None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.318542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0720641-5e3d-4796-80af-f95bd01706ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.344652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d0720641-5e3d-4796-80af-f95bd01706ea None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:22.345875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d0720641-5e3d-4796-80af-f95bd01706ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.392680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:22.406261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43e7d94c306d42dab9b3d3d2f0866a85', 'request_id': 'req-a631d122-823d-425b-8d4d-0a16db95bc90'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.415395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1039/2072] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:22.416815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfb70d-8038-4012-b454-5b457a1ce803 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.417111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfb70d-8038-4012-b454-5b457a1ce803 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.417394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bfb70d-8038-4012-b454-5b457a1ce803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.443675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-99bfb70d-8038-4012-b454-5b457a1ce803 None None] MFA Rules not processed for user `fa6f09a3984c496293fb95c80edda583`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:22.444939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-99bfb70d-8038-4012-b454-5b457a1ce803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.498199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43e7d94c306d42dab9b3d3d2f0866a85', 'request_id': 'req-a631d122-823d-425b-8d4d-0a16db95bc90'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.504998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43e7d94c306d42dab9b3d3d2f0866a85', 'request_id': 'req-a631d122-823d-425b-8d4d-0a16db95bc90'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.521436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] The token cache is being invalidate because project 43e7d94c306d42dab9b3d3d2f0866a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:22.521872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.522665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-a631d122-823d-425b-8d4d-0a16db95bc90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:22.524352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1034/2073] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:22 2022] DELETE /identity/v3/projects/43e7d94c306d42dab9b3d3d2f0866a85 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:22.526741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.527127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.527501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.527842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1040/2074] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:22.530058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12520-9b95-44f5-a262-51551be948de None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.536900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12520-9b95-44f5-a262-51551be948de None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.536900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd12520-9b95-44f5-a262-51551be948de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.552435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:22.553907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6fd12520-9b95-44f5-a262-51551be948de None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:22.554032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.555881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6fd12520-9b95-44f5-a262-51551be948de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.589232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=83,keystone:INSERT=3,keystone:DELETE=14 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:22.621922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1035/2075] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:22.625612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1041/2076] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:22.625612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12520-9b95-44f5-a262-51551be948de None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.629546 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.633693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.637122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd12520-9b95-44f5-a262-51551be948de None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.651621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d244b-b71f-4692-9f0e-307f9c9e7068 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.652841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.653539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.653816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.654088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.654815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:22.655117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.655336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.656216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.656579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.656928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.657028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c61843-d26c-426a-b254-cd9753982283 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.657746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.659728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.669022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1042/2077] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:22.670589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.670880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.671160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.689641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:22.690873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:22.699553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1043/2078] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:22.700314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.709278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fa8a9-8260-4fd9-ac7d-850db06935ed None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.714326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.714939 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.715219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.715475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.716200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.718088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.778961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1036/2079] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:22.779819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.841117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38510e07-b2e9-459b-bdd8-b9d9965ed135 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.845849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.846527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.846813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.847095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.847837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:22.848145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.849738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.862580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1044/2080] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:22.862580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.862890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1037/2081] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:22.864090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:22.874175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd34d3-2fe8-48da-9ef4-65f63cc08081 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.923764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddde7bb-2aaa-40bc-acfb-6d10560a9383 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:22.928797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.929496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.929906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.930192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.930935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.932736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:22.938845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:22.938950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.939037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:22.939037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:22.939037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:22.939037 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:22.992856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:22.997335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1038/2082] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:22.997933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.001876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1045/2083] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/users => generated 285 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:23.002692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.010911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a262-f9f3-4e00-84b3-f71b0209c838 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.014340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e4a38-f0e0-43be-ae1e-365921727a44 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.016273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.018150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.018150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.018150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.018454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.020004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.020359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.020660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.020944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.021354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.022092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.023723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.035853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1046/2084] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:23.036717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.047256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0545798c-6c74-4a66-afe9-d8438b862044 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.052189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.052872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.053106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.053378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.054096 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.055711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.067153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1047/2085] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:23.068135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.078775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c718982-cbb8-4736-8540-bfb7e41f2e49 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.084196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.084834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.085120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.085399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.086111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.087715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.097829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1048/2086] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:23.098590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.107931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664f8ad-9887-4c72-8d61-0f65ca172350 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.112700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.113603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.113886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.114160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] PATH_INFO: `/v3/projects/98b7e0625e0d42a0b147fdd2be6939ec/users/0549f904a56a400dbe0791c9c8e96c69/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.125692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1039/2087] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:22 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:23.125692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.129551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b7e0625e0d42a0b147fdd2be6939ec, user_id=0549f904a56a400dbe0791c9c8e96c69, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.132356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.149540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:23.215853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fa147-4987-4ee2-bf57-814b11dac96d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.215853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.215853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.215853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.216224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] PATH_INFO: `/v3/projects/844a3fc51fb14888b8d7f9351918e0c7/users/77b4e92eaa18440298056b54a30a4c11/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.230904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] RBAC: Authorizing `identity:create_grant(project_id=844a3fc51fb14888b8d7f9351918e0c7, user_id=77b4e92eaa18440298056b54a30a4c11, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.239647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.249322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1049/2088] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:23 2022] PUT /identity/v3/projects/98b7e0625e0d42a0b147fdd2be6939ec/users/0549f904a56a400dbe0791c9c8e96c69/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:23.253397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.317386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a97d46-0bad-4994-b650-e54c1a3f1f5d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.322162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e07384-4098-4168-b0ba-94ae59a540cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.323459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07384-4098-4168-b0ba-94ae59a540cb admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.323773 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07384-4098-4168-b0ba-94ae59a540cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.324049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e07384-4098-4168-b0ba-94ae59a540cb admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.324764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07384-4098-4168-b0ba-94ae59a540cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.326739 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e07384-4098-4168-b0ba-94ae59a540cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.378880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1040/2089] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:23 2022] PUT /identity/v3/projects/844a3fc51fb14888b8d7f9351918e0c7/users/77b4e92eaa18440298056b54a30a4c11/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:23.389686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.453405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:23.460548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1050/2090] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:23 2022] POST /identity/v3/users => generated 287 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:23.462376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43c33-b395-4793-9dd2-c181fc54f08c None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.463264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43c33-b395-4793-9dd2-c181fc54f08c None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.464545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43c33-b395-4793-9dd2-c181fc54f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.469019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972196a-4cfc-48dd-9da0-bfd9832c020b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.474231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.475014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.475128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.475310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.476092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.477765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.487977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1041/2091] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:23.488782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.492368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d7a43c33-b395-4793-9dd2-c181fc54f08c None None] MFA Rules not processed for user `77b4e92eaa18440298056b54a30a4c11`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:23.495886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d7a43c33-b395-4793-9dd2-c181fc54f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:23.500208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68705e1a-2b89-4be3-b727-6f71bb43baf0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.504964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.505643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.505943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.506357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.507076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.508690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.528925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1042/2092] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:23.528925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.533368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b868d4b5-2b47-4907-ad66-17b6bbd92e5b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.541091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.542380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.542670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.542998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] PATH_INFO: `/v3/projects/98b7e0625e0d42a0b147fdd2be6939ec/users/0549f904a56a400dbe0791c9c8e96c69/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.545221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b7e0625e0d42a0b147fdd2be6939ec, user_id=0549f904a56a400dbe0791c9c8e96c69, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.548072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.564136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1043/2093] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:23 2022] PUT /identity/v3/projects/98b7e0625e0d42a0b147fdd2be6939ec/users/0549f904a56a400dbe0791c9c8e96c69/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:23.565812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.582616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1051/2094] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:23.584361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.584878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.585215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.618127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] MFA Rules not processed for user `0549f904a56a400dbe0791c9c8e96c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:23.619347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:23.646045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804b455-5e45-4327-a9ca-24a900a8f179 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.651128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.653359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.653656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.654083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.654918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.656864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.672276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1044/2095] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:23.673247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.675113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:23.684928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:23.716952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1052/2096] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:23 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:23.718522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.731355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fde7c-e815-4176-8703-428e02db34b2 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.741442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.764705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.764705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.764705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] PATH_INFO: `/v3/projects/831f42cd36b0443e8fa6685c5ff8d100/users/ae53d5ff15c840b2b2e6576da8358661/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.765253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0a821-36fd-49e8-914d-6dd9aadd9389 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.765253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.765253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.765253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.765960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.767008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=831f42cd36b0443e8fa6685c5ff8d100, user_id=ae53d5ff15c840b2b2e6576da8358661, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.771069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.826024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.827481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.838442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1045/2097] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:23.952155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:23.952155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:23.952155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42b791-57a2-4357-9dfc-121655c137e1 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:23.952714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88759dd8-9b13-4e80-8ce4-a04b62c4c629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:23.952714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759dd8-9b13-4e80-8ce4-a04b62c4c629 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:23.952714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759dd8-9b13-4e80-8ce4-a04b62c4c629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:23.952714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88759dd8-9b13-4e80-8ce4-a04b62c4c629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:23.953329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88759dd8-9b13-4e80-8ce4-a04b62c4c629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:23.953329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88759dd8-9b13-4e80-8ce4-a04b62c4c629 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:23.953329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1046/2098] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:23 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:24.075561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1053/2099] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:23 2022] PUT /identity/v3/projects/831f42cd36b0443e8fa6685c5ff8d100/users/ae53d5ff15c840b2b2e6576da8358661/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:24.080666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:24.080957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:24.081240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:24.099005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:24.100243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:24.169974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1047/2100] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:24.182423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:24.182423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:24.249841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:24.253895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae6ba2-7430-4181-bbdd-f3bdfc7b1496 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:24.257833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:24.258472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:24.258757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:24.259031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:24.268564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:24.269985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:24.280091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1054/2101] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:24.311645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:24.364264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:24.369389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f642bf9-fbd5-4447-a9ba-a9ccb48aa12a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:24.373143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:24.378109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:24.378109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:24.378109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:24.378109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:24.442135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:24.443848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1addc-4ce8-4224-9a89-79e98bb8a0ad service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:24.453645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1048/2102] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:24 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:25.032403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:25.034036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:25.048067 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a974d220-0839-4903-bbe3-be9a6005117c service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:25.052817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:25.053689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:25.053966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:25.054236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] PATH_INFO: `/v3/users/1e9b7945b254480ebaa521ac09118522` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:25.055422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e9b7945b254480ebaa521ac09118522)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:25.058124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:25.270749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e9b7945b254480ebaa521ac09118522'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:25.271782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e9b7945b254480ebaa521ac09118522'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:25.372430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e9b7945b254480ebaa521ac09118522'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:25.395452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e9b7945b254480ebaa521ac09118522'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:25.404843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e9b7945b254480ebaa521ac09118522'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:25.415979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1055/2103] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:25 2022] DELETE /identity/v3/users/1e9b7945b254480ebaa521ac09118522 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:25.419576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb38e-0fda-4c23-b7c9-a1feceab2d86 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:25.419576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb38e-0fda-4c23-b7c9-a1feceab2d86 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:25.419576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fb38e-0fda-4c23-b7c9-a1feceab2d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:25.437100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-2a9fb38e-0fda-4c23-b7c9-a1feceab2d86 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:25.438329 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2a9fb38e-0fda-4c23-b7c9-a1feceab2d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:25.499940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1049/2104] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:25.509698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:25.516779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:25.571292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03956c-b02d-4fea-af99-27e2e30c5e91 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:25.576600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:25.576600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:25.576600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:25.576890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:25.585695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:25.587154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:25.597488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1056/2105] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:25.849617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:25.849909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:25.850188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:25.898967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] MFA Rules not processed for user `ae53d5ff15c840b2b2e6576da8358661`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:25.900443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:25.999733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1050/2106] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:26.002162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.005192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:26.032108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb4a59c-b195-4baa-98b6-eeebb4c003d5 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.039921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.041029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.041562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.042056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.043305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.047019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.082884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.086146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:26.105762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62883a-15b1-413e-942a-8b76eb4a3920 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.115560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.115931 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.116162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.116551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] PATH_INFO: `/v3/projects/8518347e59ff4feea367cca79c4ac0e0` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.117687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8518347e59ff4feea367cca79c4ac0e0)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.120326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.206530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1057/2107] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:26 2022] POST /identity/v3/projects => generated 371 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:26.211990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.221511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:26.261507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8518347e59ff4feea367cca79c4ac0e0', 'request_id': 'req-f05cd107-548a-4bbb-bb24-0d9e850ec70d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.286703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:26.306450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2f8e3-b429-4bc2-9f88-9b7c8b305708 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.327772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8518347e59ff4feea367cca79c4ac0e0', 'request_id': 'req-f05cd107-548a-4bbb-bb24-0d9e850ec70d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.346328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8518347e59ff4feea367cca79c4ac0e0', 'request_id': 'req-f05cd107-548a-4bbb-bb24-0d9e850ec70d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.367071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] The token cache is being invalidate because project 8518347e59ff4feea367cca79c4ac0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:26.367474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.368583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.371549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1051/2108] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:26 2022] DELETE /identity/v3/projects/8518347e59ff4feea367cca79c4ac0e0 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:26.375741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.378115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:26.421593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.423118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.423398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.423685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.424378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.426341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.459781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cd107-548a-4bbb-bb24-0d9e850ec70d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.468407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.469305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.469631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.469923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] PATH_INFO: `/v3/users/50df0e2448cb45aaac0d269f4e76a9d3` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.471216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50df0e2448cb45aaac0d269f4e76a9d3)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.473257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.521376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1058/2109] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:26 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:26.526896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.528506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:26.540026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fed99e-35f5-49e2-8d5b-1a3630f9797c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.545455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.546109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.546393 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.546736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.547461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.549391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.559743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1059/2110] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:26.563906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.575810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0d02b-dadb-43ab-ba58-44f0a7f79b8e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.586071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.586938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.587226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.587531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] PATH_INFO: `/v3/projects/eb8aaf4b8abe43ee9d59883067a2a7bb/users/cd549823cf984406ac8ef2b857c8345a/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.602832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8aaf4b8abe43ee9d59883067a2a7bb, user_id=cd549823cf984406ac8ef2b857c8345a, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.606269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.705061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50df0e2448cb45aaac0d269f4e76a9d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.705716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50df0e2448cb45aaac0d269f4e76a9d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.709731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1060/2111] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:26 2022] PUT /identity/v3/projects/eb8aaf4b8abe43ee9d59883067a2a7bb/users/cd549823cf984406ac8ef2b857c8345a/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:26.718148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.718148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:26.784009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b1241d-a013-4e0b-84ee-159054f9a5e0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.788690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.789360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.789631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.789908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.790629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.792856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.802728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1061/2112] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:26.806933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:26.816216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eab98-f0ab-4aac-9c00-dde407eeb428 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:26.817635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50df0e2448cb45aaac0d269f4e76a9d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.821134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:26.822042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:26.822308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:26.822583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] PATH_INFO: `/v3/projects/eb8aaf4b8abe43ee9d59883067a2a7bb/users/cd549823cf984406ac8ef2b857c8345a/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:26.825168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb8aaf4b8abe43ee9d59883067a2a7bb, user_id=cd549823cf984406ac8ef2b857c8345a, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:26.825879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50df0e2448cb45aaac0d269f4e76a9d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.827931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:26.837963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1062/2113] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:26 2022] PUT /identity/v3/projects/eb8aaf4b8abe43ee9d59883067a2a7bb/users/cd549823cf984406ac8ef2b857c8345a/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:26.839365 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50df0e2448cb45aaac0d269f4e76a9d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:26.845717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1052/2114] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:26 2022] DELETE /identity/v3/users/50df0e2448cb45aaac0d269f4e76a9d3 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:27.231303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:27.233292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:27.271729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:27.308381 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63120f9e-6294-4566-91d3-2110b1a31f2f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:27.366759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:27.367638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:27.367954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:27.368235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] PATH_INFO: `/v3/projects/86d772f38d0544228631df1df53973bb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:27.369478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d772f38d0544228631df1df53973bb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:27.371447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:27.493987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d772f38d0544228631df1df53973bb', 'request_id': 'req-df11f561-cfaa-47ee-96aa-b11110b891c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.629279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d772f38d0544228631df1df53973bb', 'request_id': 'req-df11f561-cfaa-47ee-96aa-b11110b891c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.636594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d772f38d0544228631df1df53973bb', 'request_id': 'req-df11f561-cfaa-47ee-96aa-b11110b891c8'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.649140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] The token cache is being invalidate because project 86d772f38d0544228631df1df53973bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:27.649475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.650130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-df11f561-cfaa-47ee-96aa-b11110b891c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.653592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1063/2115] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:27 2022] DELETE /identity/v3/projects/86d772f38d0544228631df1df53973bb => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:27.659109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:27.669120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808d628-879a-4a4d-81ff-f09ab9f355b2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:27.700312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:27.730487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:27.731368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:27.731676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:27.731972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] PATH_INFO: `/v3/users/60e38277669b4ce8b367fa29bd20ff3a` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:27.733188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60e38277669b4ce8b367fa29bd20ff3a)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:27.735123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:27.923481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60e38277669b4ce8b367fa29bd20ff3a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.924167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60e38277669b4ce8b367fa29bd20ff3a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:27.995636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60e38277669b4ce8b367fa29bd20ff3a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.004547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60e38277669b4ce8b367fa29bd20ff3a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.016734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60e38277669b4ce8b367fa29bd20ff3a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.022854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1053/2116] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:27 2022] DELETE /identity/v3/users/60e38277669b4ce8b367fa29bd20ff3a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:28.152502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:28.152717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:28.152995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:28.180071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] MFA Rules not processed for user `cd549823cf984406ac8ef2b857c8345a`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:28.181299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:28.274564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:28.293874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1064/2117] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:28 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:28.313533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:28.314166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:28.395036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c329e1-6615-459e-9d7e-6ad83c7c61d2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:28.398810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:28.403061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:28.403061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:28.403061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:28.403061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:28.465974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:28.467524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629b165-2161-466f-80eb-de96bebef625 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:28.476678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1054/2118] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:28.545472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:28.546976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:28.632504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082ccb4-c9b7-42a5-a081-87687adb3217 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:28.637447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:28.638332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:28.638616 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:28.638889 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] PATH_INFO: `/v3/projects/6d37eacb82e04bb9b3cf67f696af6529` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:28.640165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d37eacb82e04bb9b3cf67f696af6529)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:28.642111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:28.755873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d37eacb82e04bb9b3cf67f696af6529', 'request_id': 'req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.840853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d37eacb82e04bb9b3cf67f696af6529', 'request_id': 'req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.848246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d37eacb82e04bb9b3cf67f696af6529', 'request_id': 'req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.860959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] The token cache is being invalidate because project 6d37eacb82e04bb9b3cf67f696af6529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:28.861245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.861959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-22c76f55-1be1-4b88-9a5c-84fbd2ef1961 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:28.863559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1065/2119] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:28 2022] DELETE /identity/v3/projects/6d37eacb82e04bb9b3cf67f696af6529 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:28.868435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:28.868902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:28.869077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:28.881409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:28.888549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:28.888549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:28.956593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1055/2120] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:28.959188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:28.959480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:28.959788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:28.982444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:28.983654 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:28.991601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1066/2121] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:28.996130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.006662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec82379-1b50-4ebf-a2d2-2494ed7605bb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.012920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.013664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.013949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.014225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.014952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:29.015258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.016911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.031117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1056/2122] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:29.036906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.047179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2534b-ba49-4122-a487-5e07446f2ce9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.053899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.054903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.055527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.056011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.056798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.058804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.162165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1067/2123] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:29 2022] POST /identity/v3/projects => generated 367 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:29.166634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.177714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb8834-a774-4140-9db0-bce5302aefdc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.243392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.244212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.244343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.244649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.245355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.247302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.338099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1057/2124] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:29 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:29.343806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.354730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8686a9-4eb4-4122-8f43-e752c15ea051 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.359391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.360061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.360346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.360624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.361385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.362974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.373479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1068/2125] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:29.377643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.388876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69edf6a1-4e12-4ba2-9bb5-196aca9ee247 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.394154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68294e47-7645-488b-bb5f-83ba01d893b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.395013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68294e47-7645-488b-bb5f-83ba01d893b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.395298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68294e47-7645-488b-bb5f-83ba01d893b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.396859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68294e47-7645-488b-bb5f-83ba01d893b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.396859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68294e47-7645-488b-bb5f-83ba01d893b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.398403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68294e47-7645-488b-bb5f-83ba01d893b4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.412643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1058/2126] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:29.413839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.425125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daa1f3-b21f-4f13-a013-6f80c025766e admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.429661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.430482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.430763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.431040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] PATH_INFO: `/v3/projects/b43da48465414330b81e2dd893acf5bd/users/c5668ade230c4852b29d4eb9f33fd5c2/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.446588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b43da48465414330b81e2dd893acf5bd, user_id=c5668ade230c4852b29d4eb9f33fd5c2, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.450397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.549585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1069/2127] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:29 2022] PUT /identity/v3/projects/b43da48465414330b81e2dd893acf5bd/users/c5668ade230c4852b29d4eb9f33fd5c2/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:29.560593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.560886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.561163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.588518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] MFA Rules not processed for user `c5668ade230c4852b29d4eb9f33fd5c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:29.589754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:29.654062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1059/2128] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:29.665212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:29.671794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:29.688034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:29.726064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd7ede-8b1a-4df3-aaa2-2963bec172c5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:29.730311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:29.730965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:29.731247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:29.731517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:29.739836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:29.741271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:29.749938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1070/2129] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:33.437838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:33.437838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:33.437838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:33.490676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8ba3c-ab10-4dc9-b6b1-a681df51f673 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:33.494261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:33.494903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:33.495227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:33.495551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:33.498630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:33.600732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:33.603393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5563d8a-ca23-4344-9c6c-42ade4645d7a service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:33.616277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1060/2130] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:33 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:38.312089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:38.319033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:38.378139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f614f9b7-e0e3-4af4-9222-8cc3228c8265 service glance] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:38.382688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:38.383555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:38.383866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:38.384163 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] PATH_INFO: `/v3/users/9d1cc949807e49209db2f9c31b1543cf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:38.385325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d1cc949807e49209db2f9c31b1543cf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:38.387288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:38.614218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d1cc949807e49209db2f9c31b1543cf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:38.614861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d1cc949807e49209db2f9c31b1543cf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:38.708954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d1cc949807e49209db2f9c31b1543cf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:38.722618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d1cc949807e49209db2f9c31b1543cf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:38.729728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d1cc949807e49209db2f9c31b1543cf'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:38.736390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1071/2131] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:38 2022] DELETE /identity/v3/users/9d1cc949807e49209db2f9c31b1543cf => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:38.740886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:38.741425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:38.741773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:38.758940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:38.760161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:38.829192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1061/2132] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:38.845242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:38.846780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:38.891806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:38.913030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7514ea89-3aad-4962-bc75-5038e8fd5c75 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:38.916598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:38.917202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:38.917486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:38.917807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:38.927208 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:38.928665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:38.942143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1072/2133] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:39.408228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:39.410055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:39.425275 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8bb0b-9a84-45ac-a6b0-30bf53823459 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:39.433860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:39.434740 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:39.435778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:39.436060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] PATH_INFO: `/v3/projects/9fa000322b0e47ecbd1b7f9e01b508d1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:39.437757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa000322b0e47ecbd1b7f9e01b508d1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:39.439772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:39.664906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:39.666222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa000322b0e47ecbd1b7f9e01b508d1', 'request_id': 'req-9dc54293-75db-4e80-9623-132f4d5c5880'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:39.760378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa000322b0e47ecbd1b7f9e01b508d1', 'request_id': 'req-9dc54293-75db-4e80-9623-132f4d5c5880'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:39.766559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa000322b0e47ecbd1b7f9e01b508d1', 'request_id': 'req-9dc54293-75db-4e80-9623-132f4d5c5880'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:39.779414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] The token cache is being invalidate because project 9fa000322b0e47ecbd1b7f9e01b508d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:39.779832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:39.780669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:39.784315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1062/2134] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:39 2022] DELETE /identity/v3/projects/9fa000322b0e47ecbd1b7f9e01b508d1 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:39.788442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:39.804491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2ef16-67db-4294-abcb-a4887514e681 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:39.870656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:39.871567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:39.871888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:39.872166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] PATH_INFO: `/v3/users/2e5b2c7e2c824058b9f2cdf996041c50` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:39.874062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5b2c7e2c824058b9f2cdf996041c50)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:39.876074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:40.070745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5b2c7e2c824058b9f2cdf996041c50'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.071334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5b2c7e2c824058b9f2cdf996041c50'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.131902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5b2c7e2c824058b9f2cdf996041c50'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.146876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5b2c7e2c824058b9f2cdf996041c50'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.153676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4e2d9ae6-f6b5-473b-a62b-d564ac63e3fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5b2c7e2c824058b9f2cdf996041c50'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.160313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1073/2135] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:39 2022] DELETE /identity/v3/users/2e5b2c7e2c824058b9f2cdf996041c50 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:40.659762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:40.661320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:40.676110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc54293-75db-4e80-9623-132f4d5c5880 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:40.747254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:40.748744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:40.749018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:40.749289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] PATH_INFO: `/v3/projects/0ec27faceae44070920e5f5cdeafbf9f` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:40.750606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ec27faceae44070920e5f5cdeafbf9f)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:40.752702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:40.842519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ec27faceae44070920e5f5cdeafbf9f', 'request_id': 'req-abfb4fa8-e5e7-4213-85d7-f77fefea793e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.908899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ec27faceae44070920e5f5cdeafbf9f', 'request_id': 'req-abfb4fa8-e5e7-4213-85d7-f77fefea793e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.915530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ec27faceae44070920e5f5cdeafbf9f', 'request_id': 'req-abfb4fa8-e5e7-4213-85d7-f77fefea793e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.926737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] The token cache is being invalidate because project 0ec27faceae44070920e5f5cdeafbf9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:40.927027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.927686 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-abfb4fa8-e5e7-4213-85d7-f77fefea793e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:40.929239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1063/2136] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:40 2022] DELETE /identity/v3/projects/0ec27faceae44070920e5f5cdeafbf9f => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:40.934206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:40.934686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:40.934797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:40.964665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:40.965897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:41.061507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1074/2137] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:41.067621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.068029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.068310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.103201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:41.105549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:41.115205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1064/2138] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:41.119683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.125200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:41.130582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e03a0c-2645-4d10-b456-018403af9404 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.135627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.136399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.140721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.140721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.140721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:41.140721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.140721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.156686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1075/2139] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:41.158099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.170300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e136773-4485-484a-bd9d-8551a9f1e838 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.174981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.175747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.176095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.176376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.177066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.178957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.314280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1065/2140] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:41 2022] POST /identity/v3/projects => generated 399 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:41.315382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.325380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba907fb-98f1-4a79-bbaa-cae69f9f5cb1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.390775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.391352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.391638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.392207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.392888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.394813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.477270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1076/2141] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:41 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:41.480677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.491895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47782a40-8da9-4157-951d-1685b3070620 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.499353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.499353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.499353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.499603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.500190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.502034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.513259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1066/2142] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:41.517657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.542017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14f34a-4c33-4902-b2cf-2af0ab30300f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.547902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83402ba9-0b19-4d73-b89b-63b7c8ea4cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.548682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83402ba9-0b19-4d73-b89b-63b7c8ea4cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.551066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83402ba9-0b19-4d73-b89b-63b7c8ea4cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.551066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83402ba9-0b19-4d73-b89b-63b7c8ea4cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.551066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83402ba9-0b19-4d73-b89b-63b7c8ea4cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.551931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83402ba9-0b19-4d73-b89b-63b7c8ea4cc7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.563958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1077/2143] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:41.568466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.581665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386f471-858c-43a5-990f-abfddf4dec5e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.589902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.590875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.596998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.596998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] PATH_INFO: `/v3/projects/7665173ecf9944e9af8e4d5bf0ffc58c/users/7167171f68f148538360031a3bd317e6/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.608424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7665173ecf9944e9af8e4d5bf0ffc58c, user_id=7167171f68f148538360031a3bd317e6, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.611099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.689481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1067/2144] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:41 2022] PUT /identity/v3/projects/7665173ecf9944e9af8e4d5bf0ffc58c/users/7167171f68f148538360031a3bd317e6/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:41.692628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.692921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.693202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.727528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] MFA Rules not processed for user `7167171f68f148538360031a3bd317e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:41.727528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:41.813411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1078/2145] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:41.817694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:41.822154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:41.880497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632e3f1-da32-4acc-80ca-88364cd08491 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:41.885238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:41.885892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:41.886180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:41.886458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:41.887147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:41.889159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:41.999714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1068/2146] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:41 2022] POST /identity/v3/projects => generated 401 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:42.001536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.012414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cf9d8e-08d1-4295-991d-c5228332a920 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.071299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:42.077519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.078193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.080434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.080434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.080434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.081497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.155385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1079/2147] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:42 2022] POST /identity/v3/users => generated 307 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:42.159524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.170754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc70c7a-8662-4bde-9213-5795fc50b16f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.175300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.175985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.176279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.176537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.177278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.178895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.190234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1069/2148] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:42.198546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.209254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9280d0c-5763-4d94-8460-d1b788593579 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.214028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.214950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.215341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.215514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] PATH_INFO: `/v3/projects/86deafa1672f4b3daf78aa3739bc3d68/users/8f31734ce400402e90a30bcb6b65301d/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.233105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86deafa1672f4b3daf78aa3739bc3d68, user_id=8f31734ce400402e90a30bcb6b65301d, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.236016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.303532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1080/2149] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:42 2022] PUT /identity/v3/projects/86deafa1672f4b3daf78aa3739bc3d68/users/8f31734ce400402e90a30bcb6b65301d/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:42.307711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.317832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b37058-9a67-42cf-baf8-a1d806e9366e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.380863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421b100-6494-4f60-9f23-5bfc55eef312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.381561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b100-6494-4f60-9f23-5bfc55eef312 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.381810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b100-6494-4f60-9f23-5bfc55eef312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.382085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421b100-6494-4f60-9f23-5bfc55eef312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.382797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b100-6494-4f60-9f23-5bfc55eef312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.384418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421b100-6494-4f60-9f23-5bfc55eef312 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.394800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1070/2150] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:42 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:42.399049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.409852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ea76e-4748-498a-9ba3-e7caf4c943f4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.416868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.420019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.420019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.420019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] PATH_INFO: `/v3/projects/86deafa1672f4b3daf78aa3739bc3d68/users/8f31734ce400402e90a30bcb6b65301d/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.420346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86deafa1672f4b3daf78aa3739bc3d68, user_id=8f31734ce400402e90a30bcb6b65301d, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.422963 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.447984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1081/2151] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:42 2022] PUT /identity/v3/projects/86deafa1672f4b3daf78aa3739bc3d68/users/8f31734ce400402e90a30bcb6b65301d/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:42.453323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.453616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.453957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.482454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] MFA Rules not processed for user `8f31734ce400402e90a30bcb6b65301d`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:42.483992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:42.557055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:42.562820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1071/2152] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:42.573521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.575212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:42.642317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621abff-d846-428b-8b3c-7e90a13f93a8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.646179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.646812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.647093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.647369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.657429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.658859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.668667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1082/2153] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:42.850847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:42.852368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:42.872423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd277c-6dfb-4ef1-b5ac-87ba0c59f06a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:42.877225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a29a456-6064-423b-9505-a5450da7cf20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:42.877983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29a456-6064-423b-9505-a5450da7cf20 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:42.878182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29a456-6064-423b-9505-a5450da7cf20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:42.878491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29a456-6064-423b-9505-a5450da7cf20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:42.939691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29a456-6064-423b-9505-a5450da7cf20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:42.941219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a29a456-6064-423b-9505-a5450da7cf20 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:42.951866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1072/2154] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:42 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:49.185521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:49.253300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8b3120-03c4-4cd1-9489-355ba113cd51 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:49.257815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:49.258710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:49.258995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:49.259273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] PATH_INFO: `/v3/users/7167171f68f148538360031a3bd317e6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:49.260507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7167171f68f148538360031a3bd317e6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:49.262424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:49.482154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7167171f68f148538360031a3bd317e6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:49.482807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7167171f68f148538360031a3bd317e6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:49.548947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7167171f68f148538360031a3bd317e6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:49.550358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:49.561660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7167171f68f148538360031a3bd317e6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:49.568388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7167171f68f148538360031a3bd317e6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:49.574214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1083/2155] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:49 2022] DELETE /identity/v3/users/7167171f68f148538360031a3bd317e6 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:49.579411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:49.635088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:49.635088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:49.635088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:49.635088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:49.660461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1073/2156] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:49.670295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:49.671754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:49.729673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99f254-d8d8-48c5-8e41-86c5f53660b5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:49.735759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:49.735759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:49.735759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:49.735759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:49.744508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:49.745761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:49.759054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1084/2157] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:50.213158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:50.214657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:50.226644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41234a4-e870-48ac-b6b7-48743b0ce7d3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:50.234617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:50.235834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:50.236265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:50.236685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] PATH_INFO: `/v3/projects/7665173ecf9944e9af8e4d5bf0ffc58c` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:50.238073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7665173ecf9944e9af8e4d5bf0ffc58c)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:50.246521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:50.365934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7665173ecf9944e9af8e4d5bf0ffc58c', 'request_id': 'req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.434940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7665173ecf9944e9af8e4d5bf0ffc58c', 'request_id': 'req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.443044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7665173ecf9944e9af8e4d5bf0ffc58c', 'request_id': 'req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.455545 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] The token cache is being invalidate because project 7665173ecf9944e9af8e4d5bf0ffc58c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:50.456087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.456757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.458548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1074/2158] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:50 2022] DELETE /identity/v3/projects/7665173ecf9944e9af8e4d5bf0ffc58c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:50.462370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:50.473212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee229f-8db4-4d5e-aede-7673cbf2c2ef service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:50.533899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:50.538797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:50.538797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:50.538797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] PATH_INFO: `/v3/users/8f31734ce400402e90a30bcb6b65301d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:50.538797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f31734ce400402e90a30bcb6b65301d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:50.538797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:50.761884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f31734ce400402e90a30bcb6b65301d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.762537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f31734ce400402e90a30bcb6b65301d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.828373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f31734ce400402e90a30bcb6b65301d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.841527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f31734ce400402e90a30bcb6b65301d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.846700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:50.849498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f31734ce400402e90a30bcb6b65301d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:50.855401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1085/2159] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:50 2022] DELETE /identity/v3/users/8f31734ce400402e90a30bcb6b65301d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:51.270881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:51.272857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.292925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:51.293300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1865ac-31bc-46d2-a55c-ddf5c018f56d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:51.293813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.322330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:51.376100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:51.377064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.377352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.377633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] PATH_INFO: `/v3/projects/86deafa1672f4b3daf78aa3739bc3d68` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.379050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86deafa1672f4b3daf78aa3739bc3d68)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:51.381142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:51.381392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fce730-a48c-49d6-86d9-9b9f45237af8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:51.386133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:51.387082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.387359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.387640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] PATH_INFO: `/v3/users/77b4e92eaa18440298056b54a30a4c11` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.388856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b4e92eaa18440298056b54a30a4c11)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:51.391360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:51.505741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86deafa1672f4b3daf78aa3739bc3d68', 'request_id': 'req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.617490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86deafa1672f4b3daf78aa3739bc3d68', 'request_id': 'req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.626363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86deafa1672f4b3daf78aa3739bc3d68', 'request_id': 'req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.640913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] The token cache is being invalidate because project 86deafa1672f4b3daf78aa3739bc3d68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:51.641217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.642146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e89e79e7-1675-4d5f-b84b-9ea54e3d87d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.643267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b4e92eaa18440298056b54a30a4c11'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.644414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b4e92eaa18440298056b54a30a4c11'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.644545 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1075/2160] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:51 2022] DELETE /identity/v3/projects/86deafa1672f4b3daf78aa3739bc3d68 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:51.648932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64b6a-ee59-41ec-a66b-3c89d145fc6a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.649505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64b6a-ee59-41ec-a66b-3c89d145fc6a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.650317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee64b6a-ee59-41ec-a66b-3c89d145fc6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.669118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bee64b6a-ee59-41ec-a66b-3c89d145fc6a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:51.670324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bee64b6a-ee59-41ec-a66b-3c89d145fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.730221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1076/2161] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:51.738940 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.739231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.739506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.743103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b4e92eaa18440298056b54a30a4c11'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.756502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b4e92eaa18440298056b54a30a4c11'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.758885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:51.760112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.763313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-470fd921-33ce-4aa3-b7ed-44ac600c93fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b4e92eaa18440298056b54a30a4c11'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:51.770464 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1086/2162] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:51 2022] DELETE /identity/v3/users/77b4e92eaa18440298056b54a30a4c11 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:51.770646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1077/2163] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:51.779812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:51.780848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.781135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.781413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.789681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d1872-a90c-452b-b544-7de20e8678b6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:51.805988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:51.805988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.833949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:51.854193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:51.861874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.861874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.861874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.861874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:51.861874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:51.861874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:51.869105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1087/2164] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:51.870403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:51.872262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.882865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1078/2165] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:51.887216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:51.888958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:51.893595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa7589-57fc-4840-a10e-3c9b589fd370 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:51.898391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:51.899336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.899715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.900166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] PATH_INFO: `/v3/users/fa6f09a3984c496293fb95c80edda583` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.901459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6f09a3984c496293fb95c80edda583)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:51.903480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:51.911946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:51.951528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121fe1-989c-4278-bf63-037378a2ec25 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:51.955252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:51.955923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.956208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.956484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.965289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:51.966708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:51.976637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1079/2166] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:51.977390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:51.986778 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40533585-90c6-43e9-8432-bcb3ae846367 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:51.991411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:51.992085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:51.992649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:51.994809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:51.995514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:51.997485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.117355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1080/2167] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:51 2022] POST /identity/v3/projects => generated 389 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:52.125882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.126409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6f09a3984c496293fb95c80edda583'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.127142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6f09a3984c496293fb95c80edda583'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.194046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6f09a3984c496293fb95c80edda583'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.207231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6f09a3984c496293fb95c80edda583'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.215997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10668a7d-8350-43ab-84cf-cc9cc3582de6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6f09a3984c496293fb95c80edda583'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.223040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a240f0ca-1fd6-46d3-9ccb-3bd20979173e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.229903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1088/2168] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:51 2022] DELETE /identity/v3/users/fa6f09a3984c496293fb95c80edda583 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:52.229903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.229903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.229903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.252734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:52.253992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:52.312548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.314810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.315584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.316679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.317603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.320030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.327577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1089/2169] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:52.340732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.343925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:52.354411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:52.402761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38020f-bc95-464a-b05f-eb49db4aa8f5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.408121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.408772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.409060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.409338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.420234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.424272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.429131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1081/2170] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:52 2022] POST /identity/v3/users => generated 295 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:52.429131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.431915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:52.439462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1090/2171] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:52.440353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.441742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:52.447305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d05b9-5778-495c-a763-e912a03ec6c3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.455507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.456349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.456698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.468693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ad9cc-fd12-48fd-824a-5bf3a0ac68e5 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.469513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] PATH_INFO: `/v3/projects/844a3fc51fb14888b8d7f9351918e0c7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.472464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=844a3fc51fb14888b8d7f9351918e0c7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.474638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.484274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.485325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.485608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.485896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.486641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.488801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.508037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1091/2172] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:52.511002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.522500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb2af3-8791-4290-a55d-0e49f4c8cc52 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.532746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.533423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.533710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.533990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.534725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.536382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.547578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1092/2173] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:52.551753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.562990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f8864-7f4d-4c12-b1e9-7d6a2160de61 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.569254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1370ac3-ccab-4195-b85f-7605ed94a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.570133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370ac3-ccab-4195-b85f-7605ed94a77d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.570423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370ac3-ccab-4195-b85f-7605ed94a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.570704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1370ac3-ccab-4195-b85f-7605ed94a77d admin admin] PATH_INFO: `/v3/projects/1f9776f93d094dc7adc0e7f781171fe3/users/a5ea17859b244bb496ca519a13982f06/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.573927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:52.588931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1370ac3-ccab-4195-b85f-7605ed94a77d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f9776f93d094dc7adc0e7f781171fe3, user_id=a5ea17859b244bb496ca519a13982f06, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.591580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1370ac3-ccab-4195-b85f-7605ed94a77d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.704175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1093/2174] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:52 2022] PUT /identity/v3/projects/1f9776f93d094dc7adc0e7f781171fe3/users/a5ea17859b244bb496ca519a13982f06/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:52.710566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.710921 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.711309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.728480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844a3fc51fb14888b8d7f9351918e0c7', 'request_id': 'req-cd1cda72-9b4b-423c-b50a-92ae035055cc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.742056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] MFA Rules not processed for user `a5ea17859b244bb496ca519a13982f06`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:52.743531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:52.789950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844a3fc51fb14888b8d7f9351918e0c7', 'request_id': 'req-cd1cda72-9b4b-423c-b50a-92ae035055cc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.797413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844a3fc51fb14888b8d7f9351918e0c7', 'request_id': 'req-cd1cda72-9b4b-423c-b50a-92ae035055cc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.809820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] The token cache is being invalidate because project 844a3fc51fb14888b8d7f9351918e0c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:52.810155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.810740 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:52.812890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1082/2175] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:52 2022] DELETE /identity/v3/projects/844a3fc51fb14888b8d7f9351918e0c7 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:52.815928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1094/2176] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:52.820305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.821712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:52.895785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1cda72-9b4b-423c-b50a-92ae035055cc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:52.900933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:52.901603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:52.901889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:52.902164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:52.902869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:52.905385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:52.992892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:52.994367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:53.019158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623415ca-e956-4ac7-be41-9f98042e82fa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.020858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1083/2177] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:52 2022] POST /identity/v3/projects => generated 389 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:53.027118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.079544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.080421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.080704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.081087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] PATH_INFO: `/v3/projects/1e0ae1f2cb9245a4a0ae5f9c97260480` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.082221 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e0ae1f2cb9245a4a0ae5f9c97260480)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.084139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.096182 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c474b8-48cc-4e61-8c7e-d96b206a5c7e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.101969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.105930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.105930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.105930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.105930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.105930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.114982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:53.189598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1084/2178] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:53 2022] POST /identity/v3/users => generated 301 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:53.195688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.205868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e0ae1f2cb9245a4a0ae5f9c97260480', 'request_id': 'req-ba0749b7-8a1d-442b-b4df-f892ed56d488'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.206604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562717d-cb2b-4dce-b697-a7276c95b248 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.268667 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.269515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.269619 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.269900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.270631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.272319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.282610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1085/2179] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:53.286649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.296031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53ebd35-2b2c-481a-bdec-3a782c6fff83 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.300692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.301531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.307152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.307152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] PATH_INFO: `/v3/projects/c426d5cecd77433bb20a5c4af76022fa/users/4d89fa581ed64bc19c72ba83c43df6b1/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.316203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c426d5cecd77433bb20a5c4af76022fa, user_id=4d89fa581ed64bc19c72ba83c43df6b1, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.318851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.333051 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e0ae1f2cb9245a4a0ae5f9c97260480', 'request_id': 'req-ba0749b7-8a1d-442b-b4df-f892ed56d488'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.337551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e0ae1f2cb9245a4a0ae5f9c97260480', 'request_id': 'req-ba0749b7-8a1d-442b-b4df-f892ed56d488'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.349281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] The token cache is being invalidate because project 1e0ae1f2cb9245a4a0ae5f9c97260480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:53.349570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.350213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.351805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1095/2180] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:52 2022] DELETE /identity/v3/projects/1e0ae1f2cb9245a4a0ae5f9c97260480 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:53.356008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.357078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:53.369315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0749b7-8a1d-442b-b4df-f892ed56d488 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.374572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.375414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.375778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.376063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] PATH_INFO: `/v3/users/0549f904a56a400dbe0791c9c8e96c69` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.377263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0549f904a56a400dbe0791c9c8e96c69)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.379247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.388351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1086/2181] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:53 2022] PUT /identity/v3/projects/c426d5cecd77433bb20a5c4af76022fa/users/4d89fa581ed64bc19c72ba83c43df6b1/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:53.393222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.394818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:53.458468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bb788-712e-4b21-ad65-671311e5ecaf admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.463548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.464233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.464517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.464791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.465508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.467188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.477192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1087/2182] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:53.485228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.490376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f42f4b-c34f-4968-aeb5-a88d905af9b4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.494912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b038fd-4379-4b40-b61d-3c1426a28771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.495797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b038fd-4379-4b40-b61d-3c1426a28771 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.496078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b038fd-4379-4b40-b61d-3c1426a28771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.496363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b038fd-4379-4b40-b61d-3c1426a28771 admin admin] PATH_INFO: `/v3/projects/c426d5cecd77433bb20a5c4af76022fa/users/4d89fa581ed64bc19c72ba83c43df6b1/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.498297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b038fd-4379-4b40-b61d-3c1426a28771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c426d5cecd77433bb20a5c4af76022fa, user_id=4d89fa581ed64bc19c72ba83c43df6b1, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.501414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b038fd-4379-4b40-b61d-3c1426a28771 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.511142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1088/2183] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:53 2022] PUT /identity/v3/projects/c426d5cecd77433bb20a5c4af76022fa/users/4d89fa581ed64bc19c72ba83c43df6b1/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:53.521978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.522213 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.522605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.534765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0549f904a56a400dbe0791c9c8e96c69'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.535448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0549f904a56a400dbe0791c9c8e96c69'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.550000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] MFA Rules not processed for user `4d89fa581ed64bc19c72ba83c43df6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:53.551198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:53.577813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:53.599915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0549f904a56a400dbe0791c9c8e96c69'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.615735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0549f904a56a400dbe0791c9c8e96c69'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.619442 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1089/2184] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:53.622990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0549f904a56a400dbe0791c9c8e96c69'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:53.630349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1096/2185] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:53 2022] DELETE /identity/v3/users/0549f904a56a400dbe0791c9c8e96c69 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:53.634130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:53.635623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:53.701647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db7d24-079b-4c66-908a-3e4974136def None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:53.761086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:53.766110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:53.767132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:53.767509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:53.768392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:53.843782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:53.845242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:53.856186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1090/2186] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:54.001866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.003386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:54.084746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0449e728-ba15-4113-93cf-91d40ec5cdf2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.089975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.091083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.091580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.091890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] PATH_INFO: `/v3/users/a5ea17859b244bb496ca519a13982f06` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.093795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ea17859b244bb496ca519a13982f06)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.095789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.179644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.181246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:54.202424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61047e78-36a3-47ed-9f16-74ad8662b9f9 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.208886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.208886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.208886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.208886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] PATH_INFO: `/v3/projects/98b7e0625e0d42a0b147fdd2be6939ec` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.209887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b7e0625e0d42a0b147fdd2be6939ec)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.211907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.284506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ea17859b244bb496ca519a13982f06'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.285173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ea17859b244bb496ca519a13982f06'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.335646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:54.347692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b7e0625e0d42a0b147fdd2be6939ec', 'request_id': 'req-4abcf6a9-d7c2-4bc6-9727-433578c14763'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.377172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ea17859b244bb496ca519a13982f06'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.390375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ea17859b244bb496ca519a13982f06'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.395018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:54.397632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-37fc9f48-c2c5-4998-9b20-6fff12ed9139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ea17859b244bb496ca519a13982f06'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.403736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1097/2187] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:54 2022] DELETE /identity/v3/users/a5ea17859b244bb496ca519a13982f06 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:54.408413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.408890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.409181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.426710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:54.428050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:54.439774 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b7e0625e0d42a0b147fdd2be6939ec', 'request_id': 'req-4abcf6a9-d7c2-4bc6-9727-433578c14763'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.447209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b7e0625e0d42a0b147fdd2be6939ec', 'request_id': 'req-4abcf6a9-d7c2-4bc6-9727-433578c14763'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.460256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] The token cache is being invalidate because project 98b7e0625e0d42a0b147fdd2be6939ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:54.460551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.461679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4abcf6a9-d7c2-4bc6-9727-433578c14763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:54.462835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1091/2188] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:54 2022] DELETE /identity/v3/projects/98b7e0625e0d42a0b147fdd2be6939ec => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:54.467503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804bea5-61b3-4077-8524-6fa2a7a282a6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.467834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804bea5-61b3-4077-8524-6fa2a7a282a6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.468122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b804bea5-61b3-4077-8524-6fa2a7a282a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.492723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b804bea5-61b3-4077-8524-6fa2a7a282a6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:54.494009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b804bea5-61b3-4077-8524-6fa2a7a282a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:54.499972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1098/2189] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:54.507202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1092/2190] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:54.511156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.511498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.511818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.512256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.512583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:54.532558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:54.533790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:54.545320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1093/2191] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:54.548006 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.557849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b491d3-9baf-4df0-9ded-1c97921534ef None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.563891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.564554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.564841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.567297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.567297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:54.567297 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.567861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.575762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c9900-3356-4d55-9dff-e9e1da8a7559 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.579404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.579904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1094/2192] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:54.580083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.580368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.580643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.584101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.590313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.591777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.593587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d799b03-0015-4939-8182-4c499e1226f0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.598176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.598806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.599088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.599366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.600070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.601373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1099/2193] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:54.604419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.693905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1095/2194] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:54 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:54.698498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.708872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb754ef-fa70-4758-af03-badb219da479 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.770853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.771555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.771870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.772152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.772964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.774875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.866657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1100/2195] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:54 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:54.868271 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.880536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09cc547-1f09-43a4-b551-1ab68605a5b6 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.885767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.886479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.886775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.887058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.887853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.890190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.907081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1096/2196] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:54.911024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.924253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fad838-2124-4175-995a-0173fea28416 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.930013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1122fb3a-509e-4885-a576-94c86196cde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.930779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122fb3a-509e-4885-a576-94c86196cde6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.931128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122fb3a-509e-4885-a576-94c86196cde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.931466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122fb3a-509e-4885-a576-94c86196cde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.932295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122fb3a-509e-4885-a576-94c86196cde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.933978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1122fb3a-509e-4885-a576-94c86196cde6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:54.951023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1101/2197] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:54.956884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:54.962855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459297-3f48-41bb-af6b-7dc1fa668817 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:54.969772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:54.970742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:54.971023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:54.971298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] PATH_INFO: `/v3/projects/4cc39c7647de428b869b920e89a4b9c2/users/95059581826b4648b5f65252d9686025/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:54.992851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc39c7647de428b869b920e89a4b9c2, user_id=95059581826b4648b5f65252d9686025, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:54.995750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:55.090974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1097/2198] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:54 2022] PUT /identity/v3/projects/4cc39c7647de428b869b920e89a4b9c2/users/95059581826b4648b5f65252d9686025/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:55.093930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.094500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.096539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.139142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] MFA Rules not processed for user `95059581826b4648b5f65252d9686025`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:55.140508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:55.145113 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:55.197997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:55.199823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:55.248552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1102/2199] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:55.260761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:55.318899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6b361-f4a1-4df5-abdb-aa17f66994f3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:55.323466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:55.324352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.324639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.324920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] PATH_INFO: `/v3/projects/1f9776f93d094dc7adc0e7f781171fe3` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.326115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f9776f93d094dc7adc0e7f781171fe3)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:55.328246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:55.334557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8803b3ac-0847-4f8a-af3a-c1230eaaf76a None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:55.340421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:55.341804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.342095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.342408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.343190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:55.345157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:55.443543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:55.451672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f9776f93d094dc7adc0e7f781171fe3', 'request_id': 'req-20c67b22-e0e7-42fb-a738-669b67d7faa7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.502564 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1103/2200] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:55 2022] POST /identity/v3/projects => generated 387 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:55.504234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:55.520190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f9776f93d094dc7adc0e7f781171fe3', 'request_id': 'req-20c67b22-e0e7-42fb-a738-669b67d7faa7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.527081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f9776f93d094dc7adc0e7f781171fe3', 'request_id': 'req-20c67b22-e0e7-42fb-a738-669b67d7faa7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.539045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] The token cache is being invalidate because project 1f9776f93d094dc7adc0e7f781171fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:55.539332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.540072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.541668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1098/2201] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:55 2022] DELETE /identity/v3/projects/1f9776f93d094dc7adc0e7f781171fe3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:55.545532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:55.546905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:55.574917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdedfc4c-7794-41da-9b6e-854938b66053 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:55.580442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:55.581190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.581497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.581782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.582460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:55.584399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:55.619015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c67b22-e0e7-42fb-a738-669b67d7faa7 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:55.624427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:55.625335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.625622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.625922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] PATH_INFO: `/v3/users/4d89fa581ed64bc19c72ba83c43df6b1` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.627138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d89fa581ed64bc19c72ba83c43df6b1)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:55.629146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:55.714489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1104/2202] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:55 2022] POST /identity/v3/users => generated 300 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:55.715256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:55.716857 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:55.728795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c0de3-91b6-4ab7-a807-2a76c1641c93 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:55.735419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:55.736355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.736637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.736910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] PATH_INFO: `/v3/users/ae53d5ff15c840b2b2e6576da8358661` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.738142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae53d5ff15c840b2b2e6576da8358661)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:55.740130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:55.782844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:55.819498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d89fa581ed64bc19c72ba83c43df6b1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.820187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d89fa581ed64bc19c72ba83c43df6b1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.879770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d89fa581ed64bc19c72ba83c43df6b1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.887497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d89fa581ed64bc19c72ba83c43df6b1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.899301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d89fa581ed64bc19c72ba83c43df6b1'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.906764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1099/2203] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:55 2022] DELETE /identity/v3/users/4d89fa581ed64bc19c72ba83c43df6b1 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:55.907685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:55.911644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:55.923394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae53d5ff15c840b2b2e6576da8358661'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.924836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae53d5ff15c840b2b2e6576da8358661'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:55.980831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335e298-d329-458b-a0e9-235ef161e90b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:55.987426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:55.988412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:55.988896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:55.989399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:55.990186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:55.991883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.000277 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae53d5ff15c840b2b2e6576da8358661'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.005859 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1100/2204] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:56.006868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.009323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.017874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae53d5ff15c840b2b2e6576da8358661'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.022501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b496b-6a17-40b7-a170-cb925ecb0262 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.024897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae53d5ff15c840b2b2e6576da8358661'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.027157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.028065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.028357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.028636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] PATH_INFO: `/v3/projects/0f45fc3fc22e44d4becc805ecbbeacee/users/058ac0764a264e62a34860475e0ca3c7/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.031829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1105/2205] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:55 2022] DELETE /identity/v3/users/ae53d5ff15c840b2b2e6576da8358661 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:56.044451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f45fc3fc22e44d4becc805ecbbeacee, user_id=058ac0764a264e62a34860475e0ca3c7, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.047115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.156499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1101/2206] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:56 2022] PUT /identity/v3/projects/0f45fc3fc22e44d4becc805ecbbeacee/users/058ac0764a264e62a34860475e0ca3c7/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:56.157245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.205928 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:56.244664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d0761-c130-4a0c-8f6b-015c75e84b65 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.328380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.328380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.328380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.328380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.328933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.330581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.343522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1106/2207] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:56.346408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.366352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d289699-e6d1-4b09-9ac9-e9a45ea39847 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.371091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d176bae-700e-47e0-9919-7143a15fa6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.372130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d176bae-700e-47e0-9919-7143a15fa6a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.372317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d176bae-700e-47e0-9919-7143a15fa6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.372586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d176bae-700e-47e0-9919-7143a15fa6a5 admin admin] PATH_INFO: `/v3/projects/0f45fc3fc22e44d4becc805ecbbeacee/users/058ac0764a264e62a34860475e0ca3c7/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.374566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d176bae-700e-47e0-9919-7143a15fa6a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f45fc3fc22e44d4becc805ecbbeacee, user_id=058ac0764a264e62a34860475e0ca3c7, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.377278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d176bae-700e-47e0-9919-7143a15fa6a5 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.393058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1102/2208] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:56 2022] PUT /identity/v3/projects/0f45fc3fc22e44d4becc805ecbbeacee/users/058ac0764a264e62a34860475e0ca3c7/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:56.401792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.403608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.411449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.412215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.412215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.418397 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0ae76-87c1-4792-9cfa-b59fb8b86524 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.462129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] MFA Rules not processed for user `058ac0764a264e62a34860475e0ca3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:56.463423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.474444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:56.502500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.503306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.503591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.503890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] PATH_INFO: `/v3/projects/c426d5cecd77433bb20a5c4af76022fa` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.505130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c426d5cecd77433bb20a5c4af76022fa)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.507037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.541097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1103/2209] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:56.541954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.544811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.564823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc01258-4f60-4fb2-8de7-88c79cfbc7f6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.569970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.570852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.576409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.576409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] PATH_INFO: `/v3/projects/831f42cd36b0443e8fa6685c5ff8d100` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.576409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=831f42cd36b0443e8fa6685c5ff8d100)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.576409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.613490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c426d5cecd77433bb20a5c4af76022fa', 'request_id': 'req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.734097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '831f42cd36b0443e8fa6685c5ff8d100', 'request_id': 'req-6e0af24a-b14e-462d-8cc6-6f80379481fc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.742753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c426d5cecd77433bb20a5c4af76022fa', 'request_id': 'req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.748700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c426d5cecd77433bb20a5c4af76022fa', 'request_id': 'req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.759985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] The token cache is being invalidate because project c426d5cecd77433bb20a5c4af76022fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:56.760174 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.760829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.762389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1107/2210] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:56 2022] DELETE /identity/v3/projects/c426d5cecd77433bb20a5c4af76022fa => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:56.766370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.766370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.823386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad9035-7e9d-4e4c-b667-c4d8cbc14821 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.851031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '831f42cd36b0443e8fa6685c5ff8d100', 'request_id': 'req-6e0af24a-b14e-462d-8cc6-6f80379481fc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.858216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '831f42cd36b0443e8fa6685c5ff8d100', 'request_id': 'req-6e0af24a-b14e-462d-8cc6-6f80379481fc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.871362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] The token cache is being invalidate because project 831f42cd36b0443e8fa6685c5ff8d100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:56.871654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.872428 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e0af24a-b14e-462d-8cc6-6f80379481fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:56.873005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.873778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.874071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1104/2211] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:56 2022] DELETE /identity/v3/projects/831f42cd36b0443e8fa6685c5ff8d100 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:56.874167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.874505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.875290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096dd6-7f17-41db-808f-e3d9c46911b0 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.875624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096dd6-7f17-41db-808f-e3d9c46911b0 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.875888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d096dd6-7f17-41db-808f-e3d9c46911b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.876601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.894969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5d096dd6-7f17-41db-808f-e3d9c46911b0 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:56.896225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5d096dd6-7f17-41db-808f-e3d9c46911b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.906361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:56.941657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.943466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.954588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1108/2212] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:56.955480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:56.956854 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:56.961277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1105/2213] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:56.971699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88516e84-82fe-48dd-a78f-053070a46fd8 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:56.972999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.973690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.974277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.976747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:56.977642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:56.981843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:56.981843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] PATH_INFO: `/v3/users/cd549823cf984406ac8ef2b857c8345a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:56.981843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd549823cf984406ac8ef2b857c8345a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:56.981843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:56.997558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:56.999108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.000609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:57.009559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1106/2214] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:57.013448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.027726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a11236-5c4a-47f2-b4b5-71a732977bba None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.030450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.031379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.031379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.031693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.032920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:57.033237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.034879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.047424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1107/2215] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:43:57.048739 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.050423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.063998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43564545-5a6f-440a-bdca-3a979aa101f8 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.068707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.069570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.069886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.070173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] PATH_INFO: `/v3/users/95059581826b4648b5f65252d9686025` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.071377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95059581826b4648b5f65252d9686025)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.073359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.133420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd549823cf984406ac8ef2b857c8345a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.134101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd549823cf984406ac8ef2b857c8345a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.217932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd549823cf984406ac8ef2b857c8345a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.231037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd549823cf984406ac8ef2b857c8345a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.238801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd549823cf984406ac8ef2b857c8345a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.246105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1109/2216] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:56 2022] DELETE /identity/v3/users/cd549823cf984406ac8ef2b857c8345a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:57.246929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.248313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.273955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95059581826b4648b5f65252d9686025'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.274661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95059581826b4648b5f65252d9686025'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.318803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b579b25-6e4b-48f9-b567-f9e23bfcda8d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.323620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.324315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.324599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.324879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.325578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.327567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.344190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95059581826b4648b5f65252d9686025'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.352234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95059581826b4648b5f65252d9686025'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.365943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4c2562b3-57b9-4d93-ab69-470ea3c84fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95059581826b4648b5f65252d9686025'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.372961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1108/2217] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:57 2022] DELETE /identity/v3/users/95059581826b4648b5f65252d9686025 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:57.377890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.378292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.378580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.397672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:57.398907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.439130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1110/2218] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:57 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:57.443429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.445185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.472410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1109/2219] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:57.484278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.486066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.497579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:57.501152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:57.541661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51938c67-5148-433e-bd94-97d7c8c89c24 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.555763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.556442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.556726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.557061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.557684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.562422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.579811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79793b0-9f14-4453-947c-729fb5aa380c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.634298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.635025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.635385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.635777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.645693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.647183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.651467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1111/2220] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:57 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:43:57.651467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.651467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.657438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1110/2221] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:57.659766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.674262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc16f60-f97c-4c47-bb4a-71b868845bfa admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.679154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee01f0b-149c-4ade-a68e-59f4da8c5370 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.684337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.688398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.688398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.688603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.688695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.688695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.688852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.688852 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.689001 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.689084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] PATH_INFO: `/v3/projects/eb8aaf4b8abe43ee9d59883067a2a7bb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.691085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb8aaf4b8abe43ee9d59883067a2a7bb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.696179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.699083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1111/2222] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:57.704257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.714709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400c8b6-8549-4396-b737-7b7bad3ca242 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.721914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.721914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.721914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.721914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.722473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.724133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.735278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1112/2223] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:43:57.740029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:57.750564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62df2636-9a01-45a6-9c7e-d2260b18f13c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:57.755174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5cd3e-c270-40d3-bc4e-7eb036be0673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:57.756133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5cd3e-c270-40d3-bc4e-7eb036be0673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.756413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5cd3e-c270-40d3-bc4e-7eb036be0673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.756683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5cd3e-c270-40d3-bc4e-7eb036be0673 admin admin] PATH_INFO: `/v3/projects/0aa05f65956b4374bf2cf95e1bb9b658/users/7673534e8b8d4921b5c1334b77fdfa6f/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.771998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5cd3e-c270-40d3-bc4e-7eb036be0673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa05f65956b4374bf2cf95e1bb9b658, user_id=7673534e8b8d4921b5c1334b77fdfa6f, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:57.774682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5cd3e-c270-40d3-bc4e-7eb036be0673 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:57.854107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1113/2224] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:57 2022] PUT /identity/v3/projects/0aa05f65956b4374bf2cf95e1bb9b658/users/7673534e8b8d4921b5c1334b77fdfa6f/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:57.864245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11231de-414c-4390-8fe9-b6a4e4e1e0c1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.864604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11231de-414c-4390-8fe9-b6a4e4e1e0c1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.864961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11231de-414c-4390-8fe9-b6a4e4e1e0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:57.877064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb8aaf4b8abe43ee9d59883067a2a7bb', 'request_id': 'req-41eed60d-407f-46c4-8d65-d9cd47495f84'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.900633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c11231de-414c-4390-8fe9-b6a4e4e1e0c1 None None] MFA Rules not processed for user `7673534e8b8d4921b5c1334b77fdfa6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:57.902154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c11231de-414c-4390-8fe9-b6a4e4e1e0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:57.932086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb8aaf4b8abe43ee9d59883067a2a7bb', 'request_id': 'req-41eed60d-407f-46c4-8d65-d9cd47495f84'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.939009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb8aaf4b8abe43ee9d59883067a2a7bb', 'request_id': 'req-41eed60d-407f-46c4-8d65-d9cd47495f84'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.957693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] The token cache is being invalidate because project eb8aaf4b8abe43ee9d59883067a2a7bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:57.958002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.959776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-41eed60d-407f-46c4-8d65-d9cd47495f84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:57.963705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1112/2225] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:57 2022] DELETE /identity/v3/projects/eb8aaf4b8abe43ee9d59883067a2a7bb => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:57.972614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:57.972969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:57.973257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.002014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1114/2226] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:58.007367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:58.007367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.073139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1113/2227] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:58.082131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.082324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.082603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.119980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:58.120148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.120249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.120336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.129401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1115/2228] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:58.133128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.143557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d8016-9b81-4dd2-8f16-47c70459ad22 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.150991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:58.151878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.153130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.153512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.153917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] PATH_INFO: `/v3/projects/4cc39c7647de428b869b920e89a4b9c2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.156777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc39c7647de428b869b920e89a4b9c2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.156887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52438963-90cc-401d-94d1-24d74b2dbe45 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.159453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.161850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.162519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.162810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.163090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.163959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:58.164235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.165838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.179205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1116/2229] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:43:58.180152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.183958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.247789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e66483-15dd-4bbb-820d-6cb9d9f0ad58 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.251747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.255831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.255831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.255831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.261695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:58.262801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.267126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.275576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc39c7647de428b869b920e89a4b9c2', 'request_id': 'req-80c39f12-601c-44e4-a70c-fdb344fb1773'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.278257 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1117/2230] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:58.283003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.291044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74857c0f-682f-4a52-bd3e-f5ddaa68030b service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.355459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.356151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.356437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.356715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.357393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.359876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.360424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc39c7647de428b869b920e89a4b9c2', 'request_id': 'req-80c39f12-601c-44e4-a70c-fdb344fb1773'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.375843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc39c7647de428b869b920e89a4b9c2', 'request_id': 'req-80c39f12-601c-44e4-a70c-fdb344fb1773'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.387048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] The token cache is being invalidate because project 4cc39c7647de428b869b920e89a4b9c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:58.387346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.388052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.392092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1114/2231] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:58 2022] DELETE /identity/v3/projects/4cc39c7647de428b869b920e89a4b9c2 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:58.392092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.392092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.404537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39f12-601c-44e4-a70c-fdb344fb1773 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.409247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.410108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.410385 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.410670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] PATH_INFO: `/v3/users/7673534e8b8d4921b5c1334b77fdfa6f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.411910 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7673534e8b8d4921b5c1334b77fdfa6f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.413861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.464398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1118/2232] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:58 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:58.465222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.466806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.529448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf65076-d510-449b-9b5b-9387c9cb042e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.534414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.535270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.535712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.535907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] PATH_INFO: `/v3/users/058ac0764a264e62a34860475e0ca3c7` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.537113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=058ac0764a264e62a34860475e0ca3c7)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.539045 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.637084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7673534e8b8d4921b5c1334b77fdfa6f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.637732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7673534e8b8d4921b5c1334b77fdfa6f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.748264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7673534e8b8d4921b5c1334b77fdfa6f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.761659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7673534e8b8d4921b5c1334b77fdfa6f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.762720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '058ac0764a264e62a34860475e0ca3c7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.763363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '058ac0764a264e62a34860475e0ca3c7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.785652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7673534e8b8d4921b5c1334b77fdfa6f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.791506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1115/2233] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:58 2022] DELETE /identity/v3/users/7673534e8b8d4921b5c1334b77fdfa6f => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:58.792248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.793738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.857108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdec7c9-b2e9-486d-82ce-1926ead18578 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:58.861784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:58.862362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.862642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.862911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.863519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '058ac0764a264e62a34860475e0ca3c7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.863653 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:58.865599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:58.871408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '058ac0764a264e62a34860475e0ca3c7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.876150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:58.886014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-38e12d3d-ea5d-4286-8aa3-99a35446caa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '058ac0764a264e62a34860475e0ca3c7'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:58.892884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1119/2234] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:43:58 2022] DELETE /identity/v3/users/058ac0764a264e62a34860475e0ca3c7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:43:58.894248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:58.894615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:58.894965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:58.914485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:58.915957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.976703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1120/2235] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:58.986948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.992382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:58.992607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1116/2236] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:43:58 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:43:58.995304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:58.996984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.009543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd62fa-179c-4208-b54c-3c768df463db admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.015735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.017856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.018145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.019286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.019286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.021567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.032424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1117/2237] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:59.035863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.046249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b2b3a5-14d2-4296-81ca-e0bb1807b479 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.047118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182db2d-3a98-4a20-ae53-858dcf7bf4b3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.051101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239cafe-baea-48fd-ac2f-65038a63be87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.051338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.051775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239cafe-baea-48fd-ac2f-65038a63be87 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.051875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.052080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239cafe-baea-48fd-ac2f-65038a63be87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.052175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.052356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239cafe-baea-48fd-ac2f-65038a63be87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.052469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.053076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.054665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.062392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239cafe-baea-48fd-ac2f-65038a63be87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.074755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239cafe-baea-48fd-ac2f-65038a63be87 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.074982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1118/2238] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:43:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:43:59.074982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.075149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1121/2239] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:43:59.079600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9496b64-5d32-4118-a6dc-47e68af50063 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.084504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.085388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.085673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.085968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] PATH_INFO: `/v3/projects/e9f14005ed2b429e87e8f2d623c51572/users/6c460ed6e4ac4eeb83c6b64e92d30482/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.101919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f14005ed2b429e87e8f2d623c51572, user_id=6c460ed6e4ac4eeb83c6b64e92d30482, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.104487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.229470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1119/2240] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:43:59 2022] PUT /identity/v3/projects/e9f14005ed2b429e87e8f2d623c51572/users/6c460ed6e4ac4eeb83c6b64e92d30482/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:59.240846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.241141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.241448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.272730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] MFA Rules not processed for user `6c460ed6e4ac4eeb83c6b64e92d30482`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:59.273951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.349264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1122/2241] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:43:59.356553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.359243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.418278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.420934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.421642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611913c-1335-4ec6-9562-22a1fa8ad1a4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.427503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.428435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.428809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.429115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.439077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.440562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.449870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1120/2242] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:43:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:43:59.480407 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:59.510904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef865c-cbdb-45de-af65-a99ac5ac1f75 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.516614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.519058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.524505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.525382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.525669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.525945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] PATH_INFO: `/v3/projects/0f45fc3fc22e44d4becc805ecbbeacee` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.527159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f45fc3fc22e44d4becc805ecbbeacee)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.532263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.548559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc0eca-ebe6-49a1-8a89-30e25c74b0f2 service glance] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.553838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.554834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.555246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.555632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] PATH_INFO: `/v3/projects/0aa05f65956b4374bf2cf95e1bb9b658` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.557243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa05f65956b4374bf2cf95e1bb9b658)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.559595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.671680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f45fc3fc22e44d4becc805ecbbeacee', 'request_id': 'req-095912c5-6053-444d-b2e3-c3188f61b223'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.682453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa05f65956b4374bf2cf95e1bb9b658', 'request_id': 'req-8080d779-c63e-4da1-bd60-7cf01dc8066f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.766258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f45fc3fc22e44d4becc805ecbbeacee', 'request_id': 'req-095912c5-6053-444d-b2e3-c3188f61b223'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.772690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f45fc3fc22e44d4becc805ecbbeacee', 'request_id': 'req-095912c5-6053-444d-b2e3-c3188f61b223'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.784286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] The token cache is being invalidate because project 0f45fc3fc22e44d4becc805ecbbeacee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:59.784572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.785218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-095912c5-6053-444d-b2e3-c3188f61b223 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.786896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1123/2243] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:59 2022] DELETE /identity/v3/projects/0f45fc3fc22e44d4becc805ecbbeacee => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:59.788861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:43:59.791375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d614a-df76-4752-9d4e-ba948e00b484 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.791675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d614a-df76-4752-9d4e-ba948e00b484 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.791967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d614a-df76-4752-9d4e-ba948e00b484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.810570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-845d614a-df76-4752-9d4e-ba948e00b484 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:59.812515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-845d614a-df76-4752-9d4e-ba948e00b484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.878354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1124/2244] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:59.885981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907159a-51d8-4c7d-937f-4826f4d57cc9 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.886273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907159a-51d8-4c7d-937f-4826f4d57cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.886389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa05f65956b4374bf2cf95e1bb9b658', 'request_id': 'req-8080d779-c63e-4da1-bd60-7cf01dc8066f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.886557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907159a-51d8-4c7d-937f-4826f4d57cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.894896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa05f65956b4374bf2cf95e1bb9b658', 'request_id': 'req-8080d779-c63e-4da1-bd60-7cf01dc8066f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.906231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b907159a-51d8-4c7d-937f-4826f4d57cc9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:59.907430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b907159a-51d8-4c7d-937f-4826f4d57cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.913810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] The token cache is being invalidate because project 0aa05f65956b4374bf2cf95e1bb9b658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:43:59.918470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.918470 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:43:59.918691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1125/2245] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:59.918784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1121/2246] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:43:59 2022] DELETE /identity/v3/projects/0aa05f65956b4374bf2cf95e1bb9b658 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:43:59.921801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e4933-a92a-4fbf-a5ce-0cf3b5bb7367 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.922073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e4933-a92a-4fbf-a5ce-0cf3b5bb7367 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.922351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e4933-a92a-4fbf-a5ce-0cf3b5bb7367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.922757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.924190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.936550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080d779-c63e-4da1-bd60-7cf01dc8066f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.939746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-672e4933-a92a-4fbf-a5ce-0cf3b5bb7367 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:59.940916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-672e4933-a92a-4fbf-a5ce-0cf3b5bb7367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.941062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.941699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.942107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.942392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.943116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:43:59.943421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.945044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.948834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1126/2247] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:59.957907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.958405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.958538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1122/2248] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:43:59.958625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.962467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:43:59.972961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85169f7-86ca-491f-8886-6c57281bf3f2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:43:59.977598 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:43:59.977772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:43:59.978227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:43:59.978506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:43:59.978924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:43:59.979018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:43:59.979606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:43:59.981540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:43:59.986973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1127/2249] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:43:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:43:59.993489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.000516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90e06e-bb4a-4d5d-adb5-5436671911d9 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.005344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.005988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.006266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.006543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.007252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:00.007553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.009174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.021650 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1128/2250] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:44:00.025935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.035915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f25c40-4908-4de4-8ab9-af8231cbee0d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.040588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.044617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.044617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.044617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.044617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.044617 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.093574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1123/2251] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:43:59 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:00.098029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.132530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1129/2252] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:00.136516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.157091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:00.173062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1d514-0c6f-4528-8c0e-4f3a27f02795 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.198587 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437da37-fa8c-4657-867f-40d8b26b4be4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.203800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.204518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.204802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.205081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.205772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.207700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.229746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.232234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.232234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.232234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.232234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.234532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.286140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1130/2253] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:00.290397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.300727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb1e9d3-8549-40e9-9400-fa1d8ac96a3e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.305204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.305841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.306124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.306401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.307261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.308799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.319143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1131/2254] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:00.322833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.324665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1124/2255] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/users => generated 289 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:00.328403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.342052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7812417-7958-46b8-a5c3-67929feafbc2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.343101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2bad18-efd8-4ae2-81b5-e276558f644b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.347495 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.348040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.348189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.348444 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.348708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.348828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.348996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.349278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.349563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.350140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.351252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.351806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.366202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1132/2256] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:00.366367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1125/2257] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:00.368310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.368433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.379725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b80579b-26b4-4653-b813-1f1099f0f3fa admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.384761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c76aa-ee0d-4913-ae30-29cff152d3f7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.385039 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f35168-74fb-45e4-8f8e-136b15491eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.385932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f35168-74fb-45e4-8f8e-136b15491eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.386212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f35168-74fb-45e4-8f8e-136b15491eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.386497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f35168-74fb-45e4-8f8e-136b15491eae admin admin] PATH_INFO: `/v3/projects/0b174bcc15c547bfab707a9c58916da5/users/f22fc31e98b24370bbc9f6cb82cbb17b/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.390356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.391033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.391313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.391590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.392676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.394282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.404478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f35168-74fb-45e4-8f8e-136b15491eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b174bcc15c547bfab707a9c58916da5, user_id=f22fc31e98b24370bbc9f6cb82cbb17b, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.409155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1126/2258] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:00.409561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f35168-74fb-45e4-8f8e-136b15491eae admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.410981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.420890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e5dd53-fcce-4074-b451-d93c6b352a0c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.425479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1261f-cd62-4911-bb9b-2080fe995de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.426346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1261f-cd62-4911-bb9b-2080fe995de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.426627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1261f-cd62-4911-bb9b-2080fe995de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.426903 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1261f-cd62-4911-bb9b-2080fe995de9 admin admin] PATH_INFO: `/v3/projects/cb1ef5e38e2c43c1aab28a25a25e00b1/users/e3e2365074ea46888076bc19ff8b7e7d/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.445332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1261f-cd62-4911-bb9b-2080fe995de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1ef5e38e2c43c1aab28a25a25e00b1, user_id=e3e2365074ea46888076bc19ff8b7e7d, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.448937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1261f-cd62-4911-bb9b-2080fe995de9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.492105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1133/2259] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:00 2022] PUT /identity/v3/projects/0b174bcc15c547bfab707a9c58916da5/users/f22fc31e98b24370bbc9f6cb82cbb17b/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:00.502349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.502663 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.502978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.515467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1127/2260] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:00 2022] PUT /identity/v3/projects/cb1ef5e38e2c43c1aab28a25a25e00b1/users/e3e2365074ea46888076bc19ff8b7e7d/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:44:00.529157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.529585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.529762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.530395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] MFA Rules not processed for user `f22fc31e98b24370bbc9f6cb82cbb17b`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:00.531526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:00.556258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] MFA Rules not processed for user `e3e2365074ea46888076bc19ff8b7e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:00.557447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:00.601175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1134/2261] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:00.606541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:00.612605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.613939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:00.624430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1128/2262] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:00.628847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.671360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c73e612-7bbc-4edd-badc-9de08d137cc6 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.675178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.675933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.676222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.678374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.684983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.687009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.691347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac40e7-cc1f-462d-8a18-306262787d04 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.695655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1135/2263] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:44:00.696422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.701774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.701774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.701774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.701774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.701774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:00.702527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.792357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1129/2264] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/projects => generated 379 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:00.793932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.805845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793767b-026c-43c4-92b4-6b678ca5a382 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.857732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:00.863977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.864634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.864932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.866863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.866863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.867896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.960185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1136/2265] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:00 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:00.964184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:00.974427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e275552-f2b1-477a-8e75-aaaeb20e8f4c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:00.979155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:00.980011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:00.980291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:00.980561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:00.981320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:00.982885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:00.993646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1130/2266] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:00.997562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.008068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ad74f-16b7-40e5-85a6-df0caa0ebfdd admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.012875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0442a71-5b8e-4cb0-ae27-2e1f2f921297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.013554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0442a71-5b8e-4cb0-ae27-2e1f2f921297 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.013838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0442a71-5b8e-4cb0-ae27-2e1f2f921297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.014114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0442a71-5b8e-4cb0-ae27-2e1f2f921297 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.015482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0442a71-5b8e-4cb0-ae27-2e1f2f921297 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.017409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0442a71-5b8e-4cb0-ae27-2e1f2f921297 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.027578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1137/2267] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:01.031960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.042319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759a429-d67c-4a60-98ac-d3cc12bee2eb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.047220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.048216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.048605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.049057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] PATH_INFO: `/v3/projects/2328cde6bcb64d1bbc95c9501267ec97/users/3a7f1e6c4b07477ba2e06efb1a8be442/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.078983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=2328cde6bcb64d1bbc95c9501267ec97, user_id=3a7f1e6c4b07477ba2e06efb1a8be442, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.081798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.177504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1131/2268] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:01 2022] PUT /identity/v3/projects/2328cde6bcb64d1bbc95c9501267ec97/users/3a7f1e6c4b07477ba2e06efb1a8be442/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:01.188943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.189235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.189512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.216860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] MFA Rules not processed for user `3a7f1e6c4b07477ba2e06efb1a8be442`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:01.218572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:01.267331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:01.286290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1138/2269] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:01.291063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.359185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b77ee2-c36f-4da3-8335-55d3f81360ec admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.363886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.364549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.364835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.365111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.365844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.367836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.493154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1132/2270] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:01 2022] POST /identity/v3/projects => generated 379 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:01.497158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.510073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6265c9b7-d3c9-4801-964c-4a8038943c8d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.567812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.568483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.568768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.569046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.569843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.571827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.655371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1139/2271] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:01 2022] POST /identity/v3/users => generated 296 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:01.656494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.667770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf94ee-81c5-4e71-8d13-80e4df08128b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.673486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.674152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.674440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.674722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.675456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.681243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.688350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1133/2272] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:01.692484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.702680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8754713-bc0c-4438-8476-3cc6ab23f2a9 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.707299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.708203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.708489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.708770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] PATH_INFO: `/v3/projects/a77a7e312a3e41ee893572fb855763a6/users/fdd824737836413cbc39b9ccdb8bbfec/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.725858 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77a7e312a3e41ee893572fb855763a6, user_id=fdd824737836413cbc39b9ccdb8bbfec, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.728647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.782804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1140/2273] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:01 2022] PUT /identity/v3/projects/a77a7e312a3e41ee893572fb855763a6/users/fdd824737836413cbc39b9ccdb8bbfec/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:01.786985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.797682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3318b3-fdc5-4d79-b769-2b900d3f773d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.828838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:01.852763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4af3f-96b5-4a34-b77b-89368ae4ec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.853421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4af3f-96b5-4a34-b77b-89368ae4ec34 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.853708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4af3f-96b5-4a34-b77b-89368ae4ec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.853989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4af3f-96b5-4a34-b77b-89368ae4ec34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.854739 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4af3f-96b5-4a34-b77b-89368ae4ec34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.856466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4af3f-96b5-4a34-b77b-89368ae4ec34 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.867589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1134/2274] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:01.871716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:01.882067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43be21-bb52-4913-9544-9a7ed3ee7fc9 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:01.886853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:01.887780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.888065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.888341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] PATH_INFO: `/v3/projects/a77a7e312a3e41ee893572fb855763a6/users/fdd824737836413cbc39b9ccdb8bbfec/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.890402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77a7e312a3e41ee893572fb855763a6, user_id=fdd824737836413cbc39b9ccdb8bbfec, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:01.893350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:01.903769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1141/2275] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:01 2022] PUT /identity/v3/projects/a77a7e312a3e41ee893572fb855763a6/users/fdd824737836413cbc39b9ccdb8bbfec/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:01.915232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:01.915528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:01.915996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:01.944229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] MFA Rules not processed for user `fdd824737836413cbc39b9ccdb8bbfec`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:01.945462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:02.016433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1135/2276] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:02.027823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:02.029536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:02.091731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19610e99-1dd6-4fea-a155-894b790a02cb admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:02.096060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:02.096736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:02.097020 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:02.097296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:02.106523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:02.108000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:02.117324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1142/2277] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:02.237422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:02.250959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd12cc2-e7a1-4804-8231-316a429aec1b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:02.254528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfcd85-231f-49e3-85d8-d535603ba85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:02.255157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfcd85-231f-49e3-85d8-d535603ba85b service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:02.255439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfcd85-231f-49e3-85d8-d535603ba85b service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:02.255983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfcd85-231f-49e3-85d8-d535603ba85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:02.314415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfcd85-231f-49e3-85d8-d535603ba85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:02.315880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfcd85-231f-49e3-85d8-d535603ba85b service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:02.324693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1136/2278] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:09.889464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:09.938860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:09.953000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0298e1f5-94d7-44cd-9d26-ebc8cabcd8d1 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:09.963953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:09.963953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:09.963953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:09.963953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] PATH_INFO: `/v3/users/e3e2365074ea46888076bc19ff8b7e7d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:09.970026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e2365074ea46888076bc19ff8b7e7d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:09.970026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:10.175293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e2365074ea46888076bc19ff8b7e7d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:10.176269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e2365074ea46888076bc19ff8b7e7d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:10.252467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e2365074ea46888076bc19ff8b7e7d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:10.259878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e2365074ea46888076bc19ff8b7e7d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:10.275863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e2365074ea46888076bc19ff8b7e7d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:10.282910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1143/2279] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:09 2022] DELETE /identity/v3/users/e3e2365074ea46888076bc19ff8b7e7d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:10.287493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:10.287999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:10.288090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:10.307731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:10.308969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:10.361612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:10.373703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1137/2280] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:10.383498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:10.384948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:10.447346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b81f5-8d24-43bf-aa79-372eaa756481 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:10.450999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:10.451603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:10.451954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:10.452376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:10.462137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:10.463717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:10.485598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1144/2281] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:10.966034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:10.967472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:10.980227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4f26f-0f1d-462d-ad29-122af5d873ae None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:10.986718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:10.986718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:10.986718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:10.986718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] PATH_INFO: `/v3/projects/cb1ef5e38e2c43c1aab28a25a25e00b1` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:10.988076 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1ef5e38e2c43c1aab28a25a25e00b1)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:10.990001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:11.144147 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1ef5e38e2c43c1aab28a25a25e00b1', 'request_id': 'req-b483c89b-5ed8-40b1-b964-0f63326123d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.210979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1ef5e38e2c43c1aab28a25a25e00b1', 'request_id': 'req-b483c89b-5ed8-40b1-b964-0f63326123d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.216595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1ef5e38e2c43c1aab28a25a25e00b1', 'request_id': 'req-b483c89b-5ed8-40b1-b964-0f63326123d1'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.228130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] The token cache is being invalidate because project cb1ef5e38e2c43c1aab28a25a25e00b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:11.228419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.229055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.230593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1138/2282] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:10 2022] DELETE /identity/v3/projects/cb1ef5e38e2c43c1aab28a25a25e00b1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:11.235464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:11.245265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87316dba-3242-4980-a5e9-f417c8de88b1 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:11.300247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:11.301113 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:11.301388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:11.301666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] PATH_INFO: `/v3/users/3a7f1e6c4b07477ba2e06efb1a8be442` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:11.302829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7f1e6c4b07477ba2e06efb1a8be442)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:11.304776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:11.314604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:11.487145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7f1e6c4b07477ba2e06efb1a8be442'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.487863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7f1e6c4b07477ba2e06efb1a8be442'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.542363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7f1e6c4b07477ba2e06efb1a8be442'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.550741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7f1e6c4b07477ba2e06efb1a8be442'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.564301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7f1e6c4b07477ba2e06efb1a8be442'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:11.573851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1145/2283] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:11 2022] DELETE /identity/v3/users/3a7f1e6c4b07477ba2e06efb1a8be442 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:44:11.983537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:11.986303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:12.004885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c89b-5ed8-40b1-b964-0f63326123d1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:12.070951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:12.071845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:12.072126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:12.072400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] PATH_INFO: `/v3/projects/2328cde6bcb64d1bbc95c9501267ec97` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:12.073590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2328cde6bcb64d1bbc95c9501267ec97)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:12.075528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:12.194546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2328cde6bcb64d1bbc95c9501267ec97', 'request_id': 'req-605994e9-7c9d-49da-953f-99ad13828ab6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.268459 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2328cde6bcb64d1bbc95c9501267ec97', 'request_id': 'req-605994e9-7c9d-49da-953f-99ad13828ab6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.275354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2328cde6bcb64d1bbc95c9501267ec97', 'request_id': 'req-605994e9-7c9d-49da-953f-99ad13828ab6'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.287825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] The token cache is being invalidate because project 2328cde6bcb64d1bbc95c9501267ec97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:12.289543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.289543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.290683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1139/2284] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:11 2022] DELETE /identity/v3/projects/2328cde6bcb64d1bbc95c9501267ec97 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:12.294958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:12.296522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:12.310695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d168c3-2823-4b03-aeb2-f763647f5954 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:12.370638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:12.371523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:12.371864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:12.372177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] PATH_INFO: `/v3/users/fdd824737836413cbc39b9ccdb8bbfec` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:12.373596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdd824737836413cbc39b9ccdb8bbfec)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:12.375585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:12.573508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdd824737836413cbc39b9ccdb8bbfec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.574164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdd824737836413cbc39b9ccdb8bbfec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.662775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdd824737836413cbc39b9ccdb8bbfec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.670211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdd824737836413cbc39b9ccdb8bbfec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.683309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-af6310d9-8467-4bd9-9961-6141f1611873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdd824737836413cbc39b9ccdb8bbfec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:12.690141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1146/2285] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:12 2022] DELETE /identity/v3/users/fdd824737836413cbc39b9ccdb8bbfec => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:13.116399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:13.117960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:13.132872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605994e9-7c9d-49da-953f-99ad13828ab6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:13.140553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:13.203600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:13.203600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.203600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.203600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] PATH_INFO: `/v3/projects/a77a7e312a3e41ee893572fb855763a6` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.203942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77a7e312a3e41ee893572fb855763a6)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:13.205818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:13.332612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77a7e312a3e41ee893572fb855763a6', 'request_id': 'req-5a4a6060-d205-42ac-8131-f5033dcaf849'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:13.399601 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77a7e312a3e41ee893572fb855763a6', 'request_id': 'req-5a4a6060-d205-42ac-8131-f5033dcaf849'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:13.405850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77a7e312a3e41ee893572fb855763a6', 'request_id': 'req-5a4a6060-d205-42ac-8131-f5033dcaf849'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:13.416826 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] The token cache is being invalidate because project a77a7e312a3e41ee893572fb855763a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:13.417129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:13.417736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-5a4a6060-d205-42ac-8131-f5033dcaf849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:13.419274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1140/2286] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:13 2022] DELETE /identity/v3/projects/a77a7e312a3e41ee893572fb855763a6 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:13.426488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.427215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.427215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.441133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:13.447186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:13.448429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:13.509161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1147/2287] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:13.518728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.519019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.519297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.539213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:13.540445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:13.550845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1141/2288] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:13.555233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:13.565225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b75bea-feca-4137-8f32-a1cd860dd244 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:13.569814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:13.570465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.570748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.571025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.571766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:13.572073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:13.574644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:13.592296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1148/2289] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:44:13.594554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:13.605022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065551e-84d1-4288-a477-4ce1d50c64d9 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:13.610221 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:13.610879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.611168 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.611448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.612184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:13.614136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:13.766332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1142/2290] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:13 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:13.766483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:13.777341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172529c-8cc1-4400-a0cc-6519b7e019ee admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:13.837413 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:13.838140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.838431 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.838708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.839420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:13.841575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:13.914933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1149/2291] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:13 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:13.919190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:13.930130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bb8fc-706f-4afe-ac31-75ee6d9146c1 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:13.934762 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:13.935438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.935752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.936034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.936759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:13.938941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:13.949814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1143/2292] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:13.953683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:13.965293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dff4d73-e3db-42e9-a4a1-271962e50541 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:13.970167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad54e09-8458-42e8-a2cc-3aaf1c1bf683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:13.970828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad54e09-8458-42e8-a2cc-3aaf1c1bf683 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:13.971118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad54e09-8458-42e8-a2cc-3aaf1c1bf683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:13.971515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad54e09-8458-42e8-a2cc-3aaf1c1bf683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:13.972218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad54e09-8458-42e8-a2cc-3aaf1c1bf683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:13.974000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad54e09-8458-42e8-a2cc-3aaf1c1bf683 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:13.989705 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1150/2293] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:13.994111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:14.005326 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd1185-8799-43b2-b192-d99192cd3ec7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:14.010681 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:14.015972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:14.015972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:14.015972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] PATH_INFO: `/v3/projects/77eef5868b714f9eb0aaa809d6e4938d/users/011fa80285ed413b86dea90e96510e0f/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:14.028434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77eef5868b714f9eb0aaa809d6e4938d, user_id=011fa80285ed413b86dea90e96510e0f, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:14.031747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:14.120512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1144/2294] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:13 2022] PUT /identity/v3/projects/77eef5868b714f9eb0aaa809d6e4938d/users/011fa80285ed413b86dea90e96510e0f/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:14.133849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:14.134177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:14.134461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:14.175586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] MFA Rules not processed for user `011fa80285ed413b86dea90e96510e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:14.177369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:14.254067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:14.268477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1151/2295] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:14.279510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:14.281764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:14.328779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:14.373928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b4f24d-e82f-40ca-86e7-ffd83002f3c2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:14.379017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e8ae48-fea8-4541-8013-edae5d1d3b1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:14.379879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8ae48-fea8-4541-8013-edae5d1d3b1a service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:14.379990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8ae48-fea8-4541-8013-edae5d1d3b1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:14.380315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e8ae48-fea8-4541-8013-edae5d1d3b1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:14.414416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8ae48-fea8-4541-8013-edae5d1d3b1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:14.416049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e8ae48-fea8-4541-8013-edae5d1d3b1a service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:14.428332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1145/2296] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:15.299048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:15.300571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:15.423471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf6160-d44b-42da-91f8-1635ea1232e1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:15.428181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:15.431646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:15.431646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:15.431646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] PATH_INFO: `/v3/users/f22fc31e98b24370bbc9f6cb82cbb17b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:15.431646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f22fc31e98b24370bbc9f6cb82cbb17b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:15.432777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:15.660980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f22fc31e98b24370bbc9f6cb82cbb17b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:15.661712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f22fc31e98b24370bbc9f6cb82cbb17b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:15.765816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f22fc31e98b24370bbc9f6cb82cbb17b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:15.774171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f22fc31e98b24370bbc9f6cb82cbb17b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:15.786503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f22fc31e98b24370bbc9f6cb82cbb17b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:15.792730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1152/2297] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:15 2022] DELETE /identity/v3/users/f22fc31e98b24370bbc9f6cb82cbb17b => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:15.797996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:15.798498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:15.798595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:15.815471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:15.816707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:15.879759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1146/2298] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:15.891234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:15.892919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:15.939069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:15.974034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2515fa6a-5c0f-45d1-b3c5-8f3fd7859139 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:15.977677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90a5bd8-4454-4a08-a6e3-40cc038577af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:15.978322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5bd8-4454-4a08-a6e3-40cc038577af service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:15.978610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5bd8-4454-4a08-a6e3-40cc038577af service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:15.978890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90a5bd8-4454-4a08-a6e3-40cc038577af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:15.988966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a5bd8-4454-4a08-a6e3-40cc038577af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:15.990484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90a5bd8-4454-4a08-a6e3-40cc038577af service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:16.000676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1153/2299] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:16.495563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:16.497256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:16.533651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf627e9-a808-4d54-8f87-f11982918a25 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:16.540025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:16.540918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:16.541202 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:16.541484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] PATH_INFO: `/v3/projects/0b174bcc15c547bfab707a9c58916da5` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:16.542996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b174bcc15c547bfab707a9c58916da5)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:16.545131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:16.704615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b174bcc15c547bfab707a9c58916da5', 'request_id': 'req-1e5d7dec-36b7-4390-9310-f9326bf06e84'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:16.794007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b174bcc15c547bfab707a9c58916da5', 'request_id': 'req-1e5d7dec-36b7-4390-9310-f9326bf06e84'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:16.800981 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b174bcc15c547bfab707a9c58916da5', 'request_id': 'req-1e5d7dec-36b7-4390-9310-f9326bf06e84'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:16.813439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] The token cache is being invalidate because project 0b174bcc15c547bfab707a9c58916da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:16.815422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:16.815422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:16.816383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1147/2300] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:16 2022] DELETE /identity/v3/projects/0b174bcc15c547bfab707a9c58916da5 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:19.184352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82140c64-405e-4e47-b537-22f5d0cb3d03 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:19.184874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82140c64-405e-4e47-b537-22f5d0cb3d03 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:19.184977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82140c64-405e-4e47-b537-22f5d0cb3d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:19.202503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-82140c64-405e-4e47-b537-22f5d0cb3d03 None None] MFA Rules not processed for user `a10ffb60631240fdb6b4a69d6d7f73e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:19.203772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-82140c64-405e-4e47-b537-22f5d0cb3d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:19.259725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1154/2301] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:19.396741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:19.398179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:19.433424 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:19.498222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5d7dec-36b7-4390-9310-f9326bf06e84 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:19.503279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:19.504254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:19.504597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:19.504947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] PATH_INFO: `/v3/users/011fa80285ed413b86dea90e96510e0f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:19.506226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=011fa80285ed413b86dea90e96510e0f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:19.508333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:19.687800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '011fa80285ed413b86dea90e96510e0f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:19.690675 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '011fa80285ed413b86dea90e96510e0f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:19.748123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '011fa80285ed413b86dea90e96510e0f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:19.761867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '011fa80285ed413b86dea90e96510e0f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:19.769933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '011fa80285ed413b86dea90e96510e0f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:19.776072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1148/2302] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:19 2022] DELETE /identity/v3/users/011fa80285ed413b86dea90e96510e0f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:19.780838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:19.781232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:19.781393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:19.806716 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:19.807942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:19.867348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1155/2303] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:19.877561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:19.882400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:19.943024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c84d0-3ca6-4d4b-aa79-6a6858f5a0cc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:19.946928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:19.947700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:19.948185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:19.948557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:19.963387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:19.963387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:19.973357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1149/2304] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:20.465077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:20.466539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:20.479013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a98fb5-5c98-4d89-a68a-22bf3e09b2f4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:20.491296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:20.491296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:20.491296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:20.491296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] PATH_INFO: `/v3/projects/77eef5868b714f9eb0aaa809d6e4938d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:20.491692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77eef5868b714f9eb0aaa809d6e4938d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:20.491692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:20.599926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77eef5868b714f9eb0aaa809d6e4938d', 'request_id': 'req-778e1429-c7a8-44a6-ae80-289f052b4bf8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:20.603402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:20.730827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77eef5868b714f9eb0aaa809d6e4938d', 'request_id': 'req-778e1429-c7a8-44a6-ae80-289f052b4bf8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:20.738209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77eef5868b714f9eb0aaa809d6e4938d', 'request_id': 'req-778e1429-c7a8-44a6-ae80-289f052b4bf8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:20.749769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] The token cache is being invalidate because project 77eef5868b714f9eb0aaa809d6e4938d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:20.750134 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:20.750790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-778e1429-c7a8-44a6-ae80-289f052b4bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:20.753502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:20.755119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:20.756270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1156/2305] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:20 2022] DELETE /identity/v3/projects/77eef5868b714f9eb0aaa809d6e4938d => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:20.764188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169def54-accf-4057-8d2f-97115679b22b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:20.764478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169def54-accf-4057-8d2f-97115679b22b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:20.764756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169def54-accf-4057-8d2f-97115679b22b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:20.770195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256b327-05a1-45b0-a44c-847535c59798 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:20.801558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:20.803578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-169def54-accf-4057-8d2f-97115679b22b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:20.804824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-169def54-accf-4057-8d2f-97115679b22b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:20.830020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:20.831047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:20.831330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:20.831603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] PATH_INFO: `/v3/users/c5668ade230c4852b29d4eb9f33fd5c2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:20.832849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5668ade230c4852b29d4eb9f33fd5c2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:20.834891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:20.864624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1157/2306] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:20.872437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:20.876521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:20.876521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:20.890238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:20.892212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:20.900000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1158/2307] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:20.904026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:20.913814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cedbac9-f3d6-40fa-8f53-d6c59896631f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:20.918401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:20.919079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:20.919309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:20.919582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:20.920338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:20.920641 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:20.922407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:20.933833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1159/2308] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:44:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:44:20.938231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:20.948062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496317a-239e-43cd-b128-ed0050a55b63 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:20.952744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:20.953376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:20.954183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:20.954463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:20.955145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:20.957199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:20.978473 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5668ade230c4852b29d4eb9f33fd5c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:20.979108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5668ade230c4852b29d4eb9f33fd5c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:21.069382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5668ade230c4852b29d4eb9f33fd5c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:21.078342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1160/2309] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:20 2022] POST /identity/v3/projects => generated 371 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:21.083141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5668ade230c4852b29d4eb9f33fd5c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:21.083698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.085342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:21.090003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b47aa981-bd6f-44bf-b78e-ef1cce02b26c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5668ade230c4852b29d4eb9f33fd5c2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:21.096664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1150/2310] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:20 2022] DELETE /identity/v3/users/c5668ade230c4852b29d4eb9f33fd5c2 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:21.103023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.103314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.103593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.119706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:21.120890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:21.150379 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f58fa-de4d-4ea0-a877-96ba99f21050 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:21.183284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1151/2311] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:21.186099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:21.193192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.194583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:21.218530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:21.219248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.219647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.220068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.220860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:21.222897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:21.257401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885cfb3-0a5a-4282-a73d-bf0c7de9b4cb None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:21.261062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:21.261707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.262018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.262295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.270301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:21.272846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:21.274471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:21.284607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1152/2312] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:44:21.339751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1161/2313] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:21 2022] POST /identity/v3/users => generated 286 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:21.344317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.354389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730a104-9598-40ab-99b3-9fe12ca345b4 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:21.359471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:21.360487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.360487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.360907 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.361652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:21.363273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:21.379705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1153/2314] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:21.380831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.397268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b834b27-17c6-4ac5-b50c-c7e18a35c1a1 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:21.397878 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39dfd2-8638-4c6a-b6a4-e1a5ed0ffc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:21.398556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39dfd2-8638-4c6a-b6a4-e1a5ed0ffc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.398838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39dfd2-8638-4c6a-b6a4-e1a5ed0ffc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.399124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39dfd2-8638-4c6a-b6a4-e1a5ed0ffc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.399896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39dfd2-8638-4c6a-b6a4-e1a5ed0ffc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:21.401502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39dfd2-8638-4c6a-b6a4-e1a5ed0ffc99 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:21.415192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1162/2315] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:21.416555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.427434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831f97d-da3b-4bea-8e55-7e4fee1fd0c2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:21.434758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:21.435804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.436122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.436354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] PATH_INFO: `/v3/projects/3da9f4ff6d6a4c58bf5a0e3cf3772ac5/users/d0fbab9028c84612a3b9d8edbf04317a/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.456747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3da9f4ff6d6a4c58bf5a0e3cf3772ac5, user_id=d0fbab9028c84612a3b9d8edbf04317a, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:21.460964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:21.622842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1154/2316] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:21 2022] PUT /identity/v3/projects/3da9f4ff6d6a4c58bf5a0e3cf3772ac5/users/d0fbab9028c84612a3b9d8edbf04317a/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:21.635342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.635926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.635926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.664427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] MFA Rules not processed for user `d0fbab9028c84612a3b9d8edbf04317a`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:21.665666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:21.739538 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1163/2317] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:21.752220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.754302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:21.814193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02422f-5094-4c32-9732-592068336b0d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:21.817871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f0e533-7aa0-4035-8f36-36989045c97a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:21.818517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e533-7aa0-4035-8f36-36989045c97a service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:21.818798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e533-7aa0-4035-8f36-36989045c97a service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:21.819078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f0e533-7aa0-4035-8f36-36989045c97a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:21.827651 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e533-7aa0-4035-8f36-36989045c97a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:21.829124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f0e533-7aa0-4035-8f36-36989045c97a service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:21.838111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1155/2318] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:44:21.949979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:21.951622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:22.034547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1eb05e-a2ac-424c-9e5b-9b2799e69722 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.039339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.040254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.040547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.040886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] PATH_INFO: `/v3/projects/b43da48465414330b81e2dd893acf5bd` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.042139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b43da48465414330b81e2dd893acf5bd)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.044177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:22.160578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:22.174457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b43da48465414330b81e2dd893acf5bd', 'request_id': 'req-c17554c6-b385-4172-a2a8-d9146bedb0c3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:22.248097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b43da48465414330b81e2dd893acf5bd', 'request_id': 'req-c17554c6-b385-4172-a2a8-d9146bedb0c3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:22.257279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b43da48465414330b81e2dd893acf5bd', 'request_id': 'req-c17554c6-b385-4172-a2a8-d9146bedb0c3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:22.271794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] The token cache is being invalidate because project b43da48465414330b81e2dd893acf5bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:22.272100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:22.273361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c17554c6-b385-4172-a2a8-d9146bedb0c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:22.279449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1164/2319] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:21 2022] DELETE /identity/v3/projects/b43da48465414330b81e2dd893acf5bd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:22.281023 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.281378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.282349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.310592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:22.311816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:22.376676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1156/2320] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:22.384382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.384679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.384963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.405044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:22.406252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:22.414724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1165/2321] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:22.418905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:22.428749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c128558-302a-4ec5-87c0-edc73ab02eaf None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.433281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.433919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.434199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.434472 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.435189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:22.435494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.437179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:22.449778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1157/2322] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:44:22.454508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:22.466028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948f241-bfd7-48d0-8d45-6c77d81f37af None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.484378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.485861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.486155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.486443 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.489408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.492320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:22.631273 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1166/2323] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:22 2022] POST /identity/v3/projects => generated 375 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:22.635016 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:22.650362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a7ff1-9b3c-407b-b07d-f278268cd2d8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.728607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:22.739761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.740729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.741009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.741279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.741977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.744169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:22.834580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1158/2324] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:22 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:22.836644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:22.851358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34129518-dde7-42b0-8f61-983c2a588975 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.858471 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.859553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.864126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.864670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.866149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.868816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:22.883127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1167/2325] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:22.893062 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:22.903543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48d36c-4e38-41f8-8c5d-d6ea5484bf22 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.908850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b2026-6b58-455a-a054-603004d566f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.909513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2026-6b58-455a-a054-603004d566f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.909951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2026-6b58-455a-a054-603004d566f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.910229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b2026-6b58-455a-a054-603004d566f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.910952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b2026-6b58-455a-a054-603004d566f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.915876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143b2026-6b58-455a-a054-603004d566f0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:22.928586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1159/2326] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:22.933380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:22.944467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56ca4-a748-4ef9-bf0a-b1ba3733f6da admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:22.949194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:22.950069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:22.950353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:22.950627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] PATH_INFO: `/v3/projects/91fc055fb3ca4e6fb5929fcb13ad2f79/users/e45ac8d23008463490238c6fea3d7dc4/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:22.968951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91fc055fb3ca4e6fb5929fcb13ad2f79, user_id=e45ac8d23008463490238c6fea3d7dc4, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:22.971819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:23.064026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1168/2327] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:22 2022] PUT /identity/v3/projects/91fc055fb3ca4e6fb5929fcb13ad2f79/users/e45ac8d23008463490238c6fea3d7dc4/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:23.074200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac395519-9509-4edb-af79-a42270e5a651 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:23.074468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac395519-9509-4edb-af79-a42270e5a651 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:23.074746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac395519-9509-4edb-af79-a42270e5a651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:23.102894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-ac395519-9509-4edb-af79-a42270e5a651 None None] MFA Rules not processed for user `e45ac8d23008463490238c6fea3d7dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:23.104160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-ac395519-9509-4edb-af79-a42270e5a651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:23.191387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1160/2328] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:23.202573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:23.211802 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:23.284829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd372c34-2a96-42c8-b023-3c09dc0b2b09 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:23.290868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:23.291539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:23.291897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:23.300218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:23.309783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:23.311455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:23.324897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1169/2329] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:25.159251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:25.159698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:25.159698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:25.182894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:25.184374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:25.268918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1161/2330] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:44:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:25.315945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:25.318064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:25.419843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50a042d-e65b-410e-921b-4888e7df4a89 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:25.425488 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086cf53-d4dd-4040-ad2c-3817be3ca034 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:25.426281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086cf53-d4dd-4040-ad2c-3817be3ca034 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:25.426526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086cf53-d4dd-4040-ad2c-3817be3ca034 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:25.426777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086cf53-d4dd-4040-ad2c-3817be3ca034 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:25.441018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086cf53-d4dd-4040-ad2c-3817be3ca034 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:25.442883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086cf53-d4dd-4040-ad2c-3817be3ca034 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:25.448139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:25.453002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1170/2331] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:44:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:44:25.767855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:25.769884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:25.929732 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5edf78c-bdde-4748-bc6d-1d8fd09f6fca None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:25.940453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:25.942951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:25.943242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:25.943523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] PATH_INFO: `/v3/users/6c460ed6e4ac4eeb83c6b64e92d30482` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:25.945075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c460ed6e4ac4eeb83c6b64e92d30482)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:25.947195 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:25.954437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:26.245026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c460ed6e4ac4eeb83c6b64e92d30482'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:26.245711 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c460ed6e4ac4eeb83c6b64e92d30482'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:26.308502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c460ed6e4ac4eeb83c6b64e92d30482'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:26.334538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c460ed6e4ac4eeb83c6b64e92d30482'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:26.345066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c460ed6e4ac4eeb83c6b64e92d30482'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:26.359942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1162/2332] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:25 2022] DELETE /identity/v3/users/6c460ed6e4ac4eeb83c6b64e92d30482 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:26.364321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:26.364795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:26.364874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:26.384669 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:26.386121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:26.485073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1171/2333] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:26.496307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:26.498411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:26.573943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98700582-25aa-469c-8dd7-8be8a11d6572 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:26.579813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46ecb6-63cb-4a18-a4ca-2fcd0665ec14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:26.580468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46ecb6-63cb-4a18-a4ca-2fcd0665ec14 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:26.580916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46ecb6-63cb-4a18-a4ca-2fcd0665ec14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:26.581025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46ecb6-63cb-4a18-a4ca-2fcd0665ec14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:26.593306 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46ecb6-63cb-4a18-a4ca-2fcd0665ec14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:26.596881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46ecb6-63cb-4a18-a4ca-2fcd0665ec14 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:26.624043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1163/2334] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:27.570591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:27.570591 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:27.597559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40c16d-f853-4866-a398-9dcd1045c34e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:27.606322 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:27.607498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:27.608066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:27.608573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] PATH_INFO: `/v3/projects/e9f14005ed2b429e87e8f2d623c51572` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:27.610867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f14005ed2b429e87e8f2d623c51572)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:27.613260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:27.752087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f14005ed2b429e87e8f2d623c51572', 'request_id': 'req-6e419599-9fe9-48b1-8d43-dcc274a03734'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:27.871956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f14005ed2b429e87e8f2d623c51572', 'request_id': 'req-6e419599-9fe9-48b1-8d43-dcc274a03734'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:28.053159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f14005ed2b429e87e8f2d623c51572', 'request_id': 'req-6e419599-9fe9-48b1-8d43-dcc274a03734'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:28.053159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] The token cache is being invalidate because project e9f14005ed2b429e87e8f2d623c51572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:28.053159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:28.053159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6e419599-9fe9-48b1-8d43-dcc274a03734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:28.053159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1172/2335] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:27 2022] DELETE /identity/v3/projects/e9f14005ed2b429e87e8f2d623c51572 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:28.053623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.053623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.053623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.053623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:28.053623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:28.053623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1164/2336] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:28.054417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.054417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.054417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.057362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:28.058580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:28.074098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1173/2337] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:28.075590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.087547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155d07d4-a87b-4e48-87ce-39516e3d16e7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:28.092716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:28.093557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.094022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.100342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.101186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:28.101507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:28.103239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:28.127972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1165/2338] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:44:28.131829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.143765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c7e7f-86e7-4fe0-b978-651b2e4ec65b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:28.148851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:28.149521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.149801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.150076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.150764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:28.152696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:28.250031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1174/2339] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:28 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:28.256551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.279558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b64df6f-d1f6-44d4-a34e-ebd30437f7fb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:28.291137 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:28.373447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:28.376908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.376908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.376908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.377946 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:28.380491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:28.470913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1166/2340] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:28 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:28.474924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.498728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51a202-2507-4d05-8b8a-6bad3210a994 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:28.513682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:28.515360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.515360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.515360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.515905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:28.525248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:28.550221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1175/2341] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:28.554760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.580008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc052be1-3ef2-43cd-801b-fdd46ff96a8a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:28.595641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4137c-a85e-4d83-ab81-affe84b768be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:28.596364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4137c-a85e-4d83-ab81-affe84b768be admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.596656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4137c-a85e-4d83-ab81-affe84b768be admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.596942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4137c-a85e-4d83-ab81-affe84b768be admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.597682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4137c-a85e-4d83-ab81-affe84b768be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:28.599293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4137c-a85e-4d83-ab81-affe84b768be admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:28.628613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.637264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1167/2342] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:28.648576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab46bd8-aa32-4fd2-8c66-9761b209f3e0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:28.664909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:28.665846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.666323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.666814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] PATH_INFO: `/v3/projects/ff088d0d94d443d888c3f30baed84fcb/users/9c904b208441449c946541df93411381/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.713284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff088d0d94d443d888c3f30baed84fcb, user_id=9c904b208441449c946541df93411381, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:28.716019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:28.805826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1176/2343] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:28 2022] PUT /identity/v3/projects/ff088d0d94d443d888c3f30baed84fcb/users/9c904b208441449c946541df93411381/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:28.836481 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:28.836742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:28.836983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:28.865927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] MFA Rules not processed for user `9c904b208441449c946541df93411381`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:28.867144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:28.945906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1168/2344] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:28.957389 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:28.962161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:28.984517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:29.025357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef8b95-b277-446e-b3a4-c3e51789eeee admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:29.029847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931a739-3a81-445e-aaca-2bfa4fd34f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:29.030511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931a739-3a81-445e-aaca-2bfa4fd34f27 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:29.032126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931a739-3a81-445e-aaca-2bfa4fd34f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:29.032126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931a739-3a81-445e-aaca-2bfa4fd34f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:29.044386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931a739-3a81-445e-aaca-2bfa4fd34f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:29.045831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931a739-3a81-445e-aaca-2bfa4fd34f27 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:29.055086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1177/2345] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:32.441976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:32.536465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc64274-aafe-4e43-a259-72647087e0e0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:32.541349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:32.543267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:32.543267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:32.543267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] PATH_INFO: `/v3/users/9c904b208441449c946541df93411381` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:32.544150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c904b208441449c946541df93411381)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:32.546078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:32.585627 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:33.019797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c904b208441449c946541df93411381'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:33.020545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c904b208441449c946541df93411381'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:33.137050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c904b208441449c946541df93411381'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:33.162609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c904b208441449c946541df93411381'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:33.177410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c904b208441449c946541df93411381'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:33.196220 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1169/2346] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:32 2022] DELETE /identity/v3/users/9c904b208441449c946541df93411381 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:33.196386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:33.196386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:33.196386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:33.208101 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:33.209279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:33.290274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1178/2347] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:33.297030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:33.299285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:33.375175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d056c-ab19-4e68-adf8-55701ea534ed admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:33.384867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:33.385703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:33.386083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:33.386456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:33.399700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:33.400357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:33.412844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1170/2348] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:34.080412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:34.081994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:34.104149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef73be-88bd-469f-8ba8-6870627e70fa None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:34.116383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:34.117270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:34.117556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:34.117830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] PATH_INFO: `/v3/projects/ff088d0d94d443d888c3f30baed84fcb` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:34.122030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff088d0d94d443d888c3f30baed84fcb)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:34.131529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:34.311031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff088d0d94d443d888c3f30baed84fcb', 'request_id': 'req-1edf638f-01fc-47fa-ab4b-60501556ddcd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:34.382616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff088d0d94d443d888c3f30baed84fcb', 'request_id': 'req-1edf638f-01fc-47fa-ab4b-60501556ddcd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:34.400395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff088d0d94d443d888c3f30baed84fcb', 'request_id': 'req-1edf638f-01fc-47fa-ab4b-60501556ddcd'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:34.417296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] The token cache is being invalidate because project ff088d0d94d443d888c3f30baed84fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:34.417840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:34.418543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1edf638f-01fc-47fa-ab4b-60501556ddcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:34.420608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1179/2349] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:34 2022] DELETE /identity/v3/projects/ff088d0d94d443d888c3f30baed84fcb => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:36.928045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:36.931965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:36.947923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81095243-aa43-49d3-b64a-9a16fdd19fd9 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:37.026297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:37.027529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:37.028011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:37.028460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] PATH_INFO: `/v3/users/e45ac8d23008463490238c6fea3d7dc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:37.029958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45ac8d23008463490238c6fea3d7dc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:37.032204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:37.254409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:37.256850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45ac8d23008463490238c6fea3d7dc4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:37.257634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45ac8d23008463490238c6fea3d7dc4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:37.325133 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45ac8d23008463490238c6fea3d7dc4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:37.340233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45ac8d23008463490238c6fea3d7dc4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:37.348731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45ac8d23008463490238c6fea3d7dc4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:37.355717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1171/2350] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:36 2022] DELETE /identity/v3/users/e45ac8d23008463490238c6fea3d7dc4 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:37.360533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:37.362747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:37.362747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:37.380154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:37.381388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:37.428350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:37.447508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1180/2351] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:37.457814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:37.459431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:37.527273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f709ec7-07ad-4c85-bf10-afb7cf7a03ae admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:37.530961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1792d-8f88-4b2c-8d54-2830859be787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:37.531874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1792d-8f88-4b2c-8d54-2830859be787 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:37.532234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1792d-8f88-4b2c-8d54-2830859be787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:37.532586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1792d-8f88-4b2c-8d54-2830859be787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:37.544055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1792d-8f88-4b2c-8d54-2830859be787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:37.545426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1792d-8f88-4b2c-8d54-2830859be787 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:37.557477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1172/2352] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:38.064313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.065883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:38.080149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87867b1-2c6f-4b2a-afde-03fc45c762f2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.090536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.091287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.091569 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.092700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] PATH_INFO: `/v3/projects/91fc055fb3ca4e6fb5929fcb13ad2f79` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.094662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=91fc055fb3ca4e6fb5929fcb13ad2f79)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.096687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.193241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91fc055fb3ca4e6fb5929fcb13ad2f79', 'request_id': 'req-73647dd2-c129-4481-a9ec-c35ec51fc6fb'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:38.244945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91fc055fb3ca4e6fb5929fcb13ad2f79', 'request_id': 'req-73647dd2-c129-4481-a9ec-c35ec51fc6fb'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:38.251100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91fc055fb3ca4e6fb5929fcb13ad2f79', 'request_id': 'req-73647dd2-c129-4481-a9ec-c35ec51fc6fb'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:38.262049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] The token cache is being invalidate because project 91fc055fb3ca4e6fb5929fcb13ad2f79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:38.262335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:38.262955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-73647dd2-c129-4481-a9ec-c35ec51fc6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:38.264737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1181/2353] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:38 2022] DELETE /identity/v3/projects/91fc055fb3ca4e6fb5929fcb13ad2f79 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:38.270240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.270707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.270913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.299853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:38.301078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:38.359088 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1173/2354] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:38.367498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.369102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.369102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.386264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:38.387465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:38.396024 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1182/2355] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:38.400115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.410591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cea5218-9809-4af7-beda-72a1bc6949c1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.415095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.415886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.416192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.416465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.417201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:38.417503 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.419081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.430483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1174/2356] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:44:38.434560 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.444128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0324120-f4cf-48b4-8cf1-5a474c624db8 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.448772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.449395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.449681 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.449959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.450635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.452602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.543943 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1183/2357] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:38 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:38.548009 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.558060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b574fd-d0dd-414c-9345-02e743703c01 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.599370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:38.614210 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.614893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.615179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.615458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.616177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.618140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.727917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1175/2358] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:38 2022] POST /identity/v3/users => generated 286 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:38.729138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.740237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfcb65e-8a9a-400f-ab25-d5249abb3c64 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.745076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.745805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.746189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.746524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.747311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.749017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.760577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1184/2359] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:38.764624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.776177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89b02d-ed99-4e78-b075-9d6869d1beac admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.782542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9287f3-8cb2-4bc1-9fb1-dd07ffcb9b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.782542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9287f3-8cb2-4bc1-9fb1-dd07ffcb9b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.782542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9287f3-8cb2-4bc1-9fb1-dd07ffcb9b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.782542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9287f3-8cb2-4bc1-9fb1-dd07ffcb9b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.782991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9287f3-8cb2-4bc1-9fb1-dd07ffcb9b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.784518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9287f3-8cb2-4bc1-9fb1-dd07ffcb9b0e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.799849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1176/2360] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:38.803440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:38.813724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd86fa4-5429-4ca3-8586-aa213bd8cdc3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:38.819338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:38.820441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.820726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.821005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] PATH_INFO: `/v3/projects/3fc5fd4594f2462fad03b033bbd2aa59/users/b13ca365980f4250820a733efe5f44e9/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.838164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fc5fd4594f2462fad03b033bbd2aa59, user_id=b13ca365980f4250820a733efe5f44e9, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:38.840851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:38.920594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1185/2361] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:38 2022] PUT /identity/v3/projects/3fc5fd4594f2462fad03b033bbd2aa59/users/b13ca365980f4250820a733efe5f44e9/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:44:38.925694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c101a-847a-4b7c-87c1-4e9dd0a5144b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:38.925982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c101a-847a-4b7c-87c1-4e9dd0a5144b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:38.926261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c101a-847a-4b7c-87c1-4e9dd0a5144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:38.943576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-369c101a-847a-4b7c-87c1-4e9dd0a5144b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:38.944814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-369c101a-847a-4b7c-87c1-4e9dd0a5144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:39.007908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1177/2362] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:39.018529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:39.020339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:39.080987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f6aab8-a0ec-4247-93dd-82309114795e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:39.084647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221d818-757e-470c-a8a5-4379f4b92059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:39.085369 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d818-757e-470c-a8a5-4379f4b92059 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:39.085722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d818-757e-470c-a8a5-4379f4b92059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:39.086070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221d818-757e-470c-a8a5-4379f4b92059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:39.096143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221d818-757e-470c-a8a5-4379f4b92059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:39.097570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5221d818-757e-470c-a8a5-4379f4b92059 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:39.107128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1186/2363] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:44.662484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ccfdb-d391-4294-8b51-c67807d73f36 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:44.662993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ccfdb-d391-4294-8b51-c67807d73f36 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:44.663231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ccfdb-d391-4294-8b51-c67807d73f36 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:44.664777 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1178/2364] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:44:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:44:44.675029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6100f-febd-4192-9dcb-7793062bc3a3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:44.675637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6100f-febd-4192-9dcb-7793062bc3a3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:44.675841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6100f-febd-4192-9dcb-7793062bc3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:44.704409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:44.774492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5be6100f-febd-4192-9dcb-7793062bc3a3 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:44.775734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5be6100f-febd-4192-9dcb-7793062bc3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:44.889353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1187/2365] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:44:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:44.909922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ccfdb-d391-4294-8b51-c67807d73f36 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:44.912863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-845ccfdb-d391-4294-8b51-c67807d73f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:45.022204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ccfdb-d391-4294-8b51-c67807d73f36 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.029216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac8e34-527f-441c-bbe4-53ff21332b8d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.029960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac8e34-527f-441c-bbe4-53ff21332b8d service glance] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.030250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac8e34-527f-441c-bbe4-53ff21332b8d service glance] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.030531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac8e34-527f-441c-bbe4-53ff21332b8d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.049697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac8e34-527f-441c-bbe4-53ff21332b8d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.052786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac8e34-527f-441c-bbe4-53ff21332b8d service glance] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.074756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1179/2366] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:45.280982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.281522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.281982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.322411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] MFA Rules not processed for user `b13ca365980f4250820a733efe5f44e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:45.323636 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:45.327031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65d8fc-b81a-44f2-81dd-75f7e1636601 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.327355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65d8fc-b81a-44f2-81dd-75f7e1636601 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.327636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65d8fc-b81a-44f2-81dd-75f7e1636601 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.330148 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1180/2367] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:44:45.335254 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.335618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.346031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.354770 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:45.355951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:45.363405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1181/2368] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:44:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:45.375262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.376718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:45.387217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c51d6-dc44-4edd-93ad-1fd1f340e6c4 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.390902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.391964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.392252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.392529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.396875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1188/2369] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:45.400318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.403593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.403729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.412012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1182/2370] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:44:45.412246 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd03721b-9207-4c57-8a1d-c6aa8489bc1f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.416547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b1d45-75de-4d18-b3d7-c3b52206087e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.417217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b1d45-75de-4d18-b3d7-c3b52206087e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.417498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b1d45-75de-4d18-b3d7-c3b52206087e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.417770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b1d45-75de-4d18-b3d7-c3b52206087e admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.418448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b1d45-75de-4d18-b3d7-c3b52206087e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.420380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b1d45-75de-4d18-b3d7-c3b52206087e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.496326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1189/2371] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:45 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:45.497443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.500900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f650e-e285-4f7a-b04c-c5f7cc00f869 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.501196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f650e-e285-4f7a-b04c-c5f7cc00f869 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.501476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f650e-e285-4f7a-b04c-c5f7cc00f869 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.502987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1190/2372] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:44:45.509215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef72487-8d1b-4819-9f6c-10a073b46744 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.509645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.509939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.510224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.531084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:45.532349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:45.545913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:45.557584 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:45.570043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.570710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.570994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.571273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.572002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.573913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.600988 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1191/2373] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:44:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:45.611935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.646123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1183/2374] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:45 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:45.650184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.661058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16203735-974e-4117-9ac7-6d28f26834a6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.665977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.667069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.667547 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.668034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.668948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.670081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c030b9-8599-4478-94d2-853a303f8aa3 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.670924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.674707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.675356 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.675934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.675934 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.681154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1184/2375] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:45.685249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.688058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.689608 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.695312 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bab9aa7-8ede-4b5e-901f-40ed2f913ff6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.701404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.702303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.702644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.702929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] PATH_INFO: `/v3/projects/9d0e68f11d444e34af4438fc65cc9f66/users/d5cda8c2b325488b8d9fec9c5a25bfcb/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.708091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1192/2376] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:44:45.721887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0e68f11d444e34af4438fc65cc9f66, user_id=d5cda8c2b325488b8d9fec9c5a25bfcb, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.724744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.789605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1185/2377] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:45 2022] PUT /identity/v3/projects/9d0e68f11d444e34af4438fc65cc9f66/users/d5cda8c2b325488b8d9fec9c5a25bfcb/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:44:45.793396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.804699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f85e18e-b202-4558-8642-3355450bd82f service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.888820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb2b95-f06d-43b9-b5d8-ac18451358f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.889581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2b95-f06d-43b9-b5d8-ac18451358f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.890232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2b95-f06d-43b9-b5d8-ac18451358f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.890327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2b95-f06d-43b9-b5d8-ac18451358f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.891158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb2b95-f06d-43b9-b5d8-ac18451358f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.893723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb2b95-f06d-43b9-b5d8-ac18451358f8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.907694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1193/2378] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:45 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:45.914298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:45.925239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39e211-bec6-4c66-853d-9ac43ba759fb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:45.930108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:45.930988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:45.931272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:45.931548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] PATH_INFO: `/v3/projects/9d0e68f11d444e34af4438fc65cc9f66/users/d5cda8c2b325488b8d9fec9c5a25bfcb/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:45.933638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d0e68f11d444e34af4438fc65cc9f66, user_id=d5cda8c2b325488b8d9fec9c5a25bfcb, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:45.936342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:45.949080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1186/2379] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:45 2022] PUT /identity/v3/projects/9d0e68f11d444e34af4438fc65cc9f66/users/d5cda8c2b325488b8d9fec9c5a25bfcb/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:51.369503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814fa15-6360-4e4b-8f00-d7fa6a232f60 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:51.370021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814fa15-6360-4e4b-8f00-d7fa6a232f60 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:51.370265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5814fa15-6360-4e4b-8f00-d7fa6a232f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:51.391137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5814fa15-6360-4e4b-8f00-d7fa6a232f60 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:51.392773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5814fa15-6360-4e4b-8f00-d7fa6a232f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:51.448668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:51.457230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1194/2380] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:44:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:51.470814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:51.472995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:51.555982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667c6329-0c0f-4df1-806e-49a1e97ddeef admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:51.561361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75738480-5eab-4b40-891f-8156934102ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:51.562131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738480-5eab-4b40-891f-8156934102ca service swift] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:51.562494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738480-5eab-4b40-891f-8156934102ca service swift] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:51.562846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75738480-5eab-4b40-891f-8156934102ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:51.572340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75738480-5eab-4b40-891f-8156934102ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:51.574291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75738480-5eab-4b40-891f-8156934102ca service swift] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:51.595890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1187/2381] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:44:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:44:51.928286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:51.928652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:51.929011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:51.976990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] MFA Rules not processed for user `d5cda8c2b325488b8d9fec9c5a25bfcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:51.979930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:52.125779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1195/2382] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:52.142699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75738480-5eab-4b40-891f-8156934102ca service swift] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:52.144596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-75738480-5eab-4b40-891f-8156934102ca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:52.291003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75738480-5eab-4b40-891f-8156934102ca service swift] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:52.297022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d436cb-d38d-47f7-bc5c-9c346deccee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:52.297806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d436cb-d38d-47f7-bc5c-9c346deccee6 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:52.298340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d436cb-d38d-47f7-bc5c-9c346deccee6 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:52.298775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d436cb-d38d-47f7-bc5c-9c346deccee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:52.313659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:52.361688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d436cb-d38d-47f7-bc5c-9c346deccee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:52.363528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d436cb-d38d-47f7-bc5c-9c346deccee6 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:52.374297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1188/2383] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:54.902558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:54.904301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:55.066766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cd051-d578-4a1b-b65f-610f5e91c122 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:55.071548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:55.078747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:55.078747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:55.078747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] PATH_INFO: `/v3/users/d0fbab9028c84612a3b9d8edbf04317a` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:55.078747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0fbab9028c84612a3b9d8edbf04317a)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:55.078747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:55.736972 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0fbab9028c84612a3b9d8edbf04317a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:55.742872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0fbab9028c84612a3b9d8edbf04317a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:55.834700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0fbab9028c84612a3b9d8edbf04317a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:55.849083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0fbab9028c84612a3b9d8edbf04317a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:55.870981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0fbab9028c84612a3b9d8edbf04317a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:55.873518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:55.879926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1196/2384] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:44:54 2022] DELETE /identity/v3/users/d0fbab9028c84612a3b9d8edbf04317a => generated 0 bytes in 977 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:55.891467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc69ab0-384f-4684-b68c-7550c72a596d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:55.891467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc69ab0-384f-4684-b68c-7550c72a596d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:55.891467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc69ab0-384f-4684-b68c-7550c72a596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:55.905765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7cc69ab0-384f-4684-b68c-7550c72a596d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:55.907121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7cc69ab0-384f-4684-b68c-7550c72a596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:56.045157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1189/2385] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:56.055194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:56.056754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:56.151520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ab31f8-5e8e-4a14-b113-701c0835576e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:56.157165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:56.159557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:56.161916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:56.161916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:56.198460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:56.202410 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:56.229594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1197/2386] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:44:56.361035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25072a-91fc-45dd-8467-f5fe863d694f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:56.361035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25072a-91fc-45dd-8467-f5fe863d694f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:56.361035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25072a-91fc-45dd-8467-f5fe863d694f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:56.411708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7c25072a-91fc-45dd-8467-f5fe863d694f None None] MFA Rules not processed for user `b13ca365980f4250820a733efe5f44e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:56.411708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7c25072a-91fc-45dd-8467-f5fe863d694f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:56.497578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1190/2387] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:56.989323 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:56.990804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:57.033730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c260735-d5d9-41f6-a6c3-0a1792fbd8b1 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:57.039612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:57.046458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:57.046458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:57.046458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] PATH_INFO: `/v3/projects/3da9f4ff6d6a4c58bf5a0e3cf3772ac5` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:57.046458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3da9f4ff6d6a4c58bf5a0e3cf3772ac5)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:57.046458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:57.196784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3da9f4ff6d6a4c58bf5a0e3cf3772ac5', 'request_id': 'req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:57.256839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3da9f4ff6d6a4c58bf5a0e3cf3772ac5', 'request_id': 'req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:57.267918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3da9f4ff6d6a4c58bf5a0e3cf3772ac5', 'request_id': 'req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:57.280393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] The token cache is being invalidate because project 3da9f4ff6d6a4c58bf5a0e3cf3772ac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:44:57.280722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:57.281678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-6fe69b94-0b3e-4bf0-b95b-9833b07adc07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:44:57.283460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1198/2388] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:44:56 2022] DELETE /identity/v3/projects/3da9f4ff6d6a4c58bf5a0e3cf3772ac5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:44:57.288429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:57.288983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:57.288983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:57.309040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:57.310458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:57.386969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:57.411045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1191/2389] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:57.419447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:57.419995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:57.420368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:57.531938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:57.531938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:57.531938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1199/2390] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:44:57.532933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:57.532933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f58f1-ec84-4ade-a89c-3713689525a7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:57.532933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:57.532933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:57.533329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:57.533329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:57.533329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:44:57.533329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:57.533329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:57.533329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1192/2391] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:44:57.533954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:57.542763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99db1550-920e-4157-a1c4-a6b8b0929278 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:57.549350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:57.550260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:57.550690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:57.551064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:57.551831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:57.553797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:57.914388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1200/2392] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:44:57 2022] POST /identity/v3/projects => generated 375 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:44:57.925244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:57.935942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ca8f-e9c8-44d1-a9ae-2d8834fd49a6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:58.012490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:58.013245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:58.013591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:58.013881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:58.014585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:58.027977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:58.165633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1193/2393] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:44:57 2022] POST /identity/v3/users => generated 288 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:44:58.169604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:58.181143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b79973-032c-4bd9-842e-dafb4f33892d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:58.185534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:58.186652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:58.187274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:58.187761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:58.189678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:58.191557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:58.203937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1201/2394] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:44:58.209027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:58.219799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5269f29-312e-4c45-9822-569e2dfad535 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:58.225656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d109057-374e-409e-86f0-311d95ad4042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:58.226543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d109057-374e-409e-86f0-311d95ad4042 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:58.226832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d109057-374e-409e-86f0-311d95ad4042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:58.227211 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d109057-374e-409e-86f0-311d95ad4042 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:58.228061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d109057-374e-409e-86f0-311d95ad4042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:58.229848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d109057-374e-409e-86f0-311d95ad4042 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:58.260566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1194/2395] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:44:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:44:58.265405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:58.287353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752e6f4-1aa6-4988-8351-2911521c1f25 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:58.292221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:58.293095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:58.293376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:58.293652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] PATH_INFO: `/v3/projects/dbcd277f142a4a328f7553029beb300e/users/8d6c989fe0044f3a808481b54362c019/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:58.309248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbcd277f142a4a328f7553029beb300e, user_id=8d6c989fe0044f3a808481b54362c019, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:58.312056 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:58.399528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1202/2396] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:44:58 2022] PUT /identity/v3/projects/dbcd277f142a4a328f7553029beb300e/users/8d6c989fe0044f3a808481b54362c019/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:44:58.411612 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07bf4-fd66-473a-8e32-b9346c1f1f5d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:58.412134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07bf4-fd66-473a-8e32-b9346c1f1f5d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:58.412319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca07bf4-fd66-473a-8e32-b9346c1f1f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:58.454452 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8ca07bf4-fd66-473a-8e32-b9346c1f1f5d None None] MFA Rules not processed for user `8d6c989fe0044f3a808481b54362c019`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:44:58.455692 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8ca07bf4-fd66-473a-8e32-b9346c1f1f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:58.536330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1195/2397] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:44:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:44:58.547253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:44:58.549499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:44:58.577644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:44:58.633196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df78847-c910-47ad-be7f-65254e8c2ca8 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:44:58.637540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:44:58.641494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:44:58.641494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:44:58.641494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:44:58.649877 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:44:58.651918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:44:58.668556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1203/2398] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:44:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:45:00.770614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286289b-a046-4761-a014-e830e0737925 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:00.771235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286289b-a046-4761-a014-e830e0737925 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:00.771235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286289b-a046-4761-a014-e830e0737925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:00.789515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4286289b-a046-4761-a014-e830e0737925 None None] MFA Rules not processed for user `8d6c989fe0044f3a808481b54362c019`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:00.790630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4286289b-a046-4761-a014-e830e0737925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:00.797743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1196/2399] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:08.659060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=25 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:10.783917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:13.289698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:13.291640 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:13.388861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399347e-4fae-4e71-b2e9-aee46aa3c68e service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:13.395203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:13.398047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:13.398047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:13.398047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:13.398611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:13.411601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:13.413287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:13.423364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1204/2400] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:45:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:45:18.469240 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6658f-9a90-45cc-a18d-2adee55f24b9 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:18.470215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6658f-9a90-45cc-a18d-2adee55f24b9 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:18.470450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6658f-9a90-45cc-a18d-2adee55f24b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:18.527309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-70d6658f-9a90-45cc-a18d-2adee55f24b9 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:18.528665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70d6658f-9a90-45cc-a18d-2adee55f24b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:18.596559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1197/2401] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 20:45:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:18.609479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:18.611582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:18.707265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bfe588-bcdc-4fbb-a0da-a8043250d3dc service placement] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:18.712582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:18.713224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:18.713777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:18.713777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:18.722749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:18.724388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:18.732833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1205/2402] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 20:45:20.438908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583b390-611d-4a0f-8259-0631720437be None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:20.439416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583b390-611d-4a0f-8259-0631720437be None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:20.439621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8583b390-611d-4a0f-8259-0631720437be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:20.458028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8583b390-611d-4a0f-8259-0631720437be None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:20.459356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8583b390-611d-4a0f-8259-0631720437be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:20.466973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1198/2403] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:45:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:20.479175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:20.490082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc7394b-7b64-4312-be1b-5306e4454eba service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:20.498424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dba36-8344-4674-a165-8d445643c95f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:20.498424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dba36-8344-4674-a165-8d445643c95f service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:20.498424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dba36-8344-4674-a165-8d445643c95f service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:20.498424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509dba36-8344-4674-a165-8d445643c95f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:20.504211 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509dba36-8344-4674-a165-8d445643c95f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:20.505781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509dba36-8344-4674-a165-8d445643c95f service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:20.513910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1206/2404] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:45:21.576360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec4bd6-02b8-4389-a205-52c765971642 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:21.576842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec4bd6-02b8-4389-a205-52c765971642 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:21.576842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec4bd6-02b8-4389-a205-52c765971642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:21.594212 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-beec4bd6-02b8-4389-a205-52c765971642 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:21.595447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-beec4bd6-02b8-4389-a205-52c765971642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:21.602485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1199/2405] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 20:45:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:21.615124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dba36-8344-4674-a165-8d445643c95f service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:21.625709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509dba36-8344-4674-a165-8d445643c95f service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:21.632034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:21.632676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:21.632966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:21.633249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:21.643470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:21.645827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:21.655748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1207/2406] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:45:22.323198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a736-03bd-4bb3-b07e-ef051529de6d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:22.323489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a736-03bd-4bb3-b07e-ef051529de6d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:22.327246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0a736-03bd-4bb3-b07e-ef051529de6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:22.343775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7fd0a736-03bd-4bb3-b07e-ef051529de6d None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:22.345904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7fd0a736-03bd-4bb3-b07e-ef051529de6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:22.353109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1200/2407] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:45:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:22.365087 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:22.375506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15990-a758-41c2-b87e-55730556ae59 service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:22.380633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:22.381331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:22.381622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:22.381923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:22.390867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:22.393331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:22.399129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:22.403427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1208/2408] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:45:22.971231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:22.971703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:22.971823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:22.989885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:22.991127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:22.999793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1201/2409] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 20:45:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:23.012147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:23.023277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ef20b1-4373-41b7-ab32-a487fbc26486 service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:23.028536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea67d4-21ae-4724-9003-ed58c1b713bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:23.029168 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea67d4-21ae-4724-9003-ed58c1b713bf service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:23.029449 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea67d4-21ae-4724-9003-ed58c1b713bf service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:23.029723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea67d4-21ae-4724-9003-ed58c1b713bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:23.038517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea67d4-21ae-4724-9003-ed58c1b713bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:23.040109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea67d4-21ae-4724-9003-ed58c1b713bf service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:23.048047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1209/2410] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:45:25.515609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:25.577874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e893afa-8369-44eb-a8a6-f6811c4e42cc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:25.582871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:25.583759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:25.584060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:25.584322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] PATH_INFO: `/v3/users/8d6c989fe0044f3a808481b54362c019` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:25.585501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d6c989fe0044f3a808481b54362c019)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:25.587443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:25.772315 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d6c989fe0044f3a808481b54362c019'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:25.772878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d6c989fe0044f3a808481b54362c019'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:25.774577 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:25.849697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d6c989fe0044f3a808481b54362c019'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:25.861448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d6c989fe0044f3a808481b54362c019'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:25.867987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d6c989fe0044f3a808481b54362c019'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:25.873800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1202/2411] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:45:25 2022] DELETE /identity/v3/users/8d6c989fe0044f3a808481b54362c019 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:45:25.878876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:25.879380 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:25.879472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:25.897485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:25.898675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:25.955187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1210/2412] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:45:25.964758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:25.966185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:26.023912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92b21b-a655-4c55-810b-c660e698cff6 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:26.027752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29223703-d58b-4dde-b5d0-2a6068e4799e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:26.028396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29223703-d58b-4dde-b5d0-2a6068e4799e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.028970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29223703-d58b-4dde-b5d0-2a6068e4799e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.028970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29223703-d58b-4dde-b5d0-2a6068e4799e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.038916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29223703-d58b-4dde-b5d0-2a6068e4799e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:26.040311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29223703-d58b-4dde-b5d0-2a6068e4799e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:26.049688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1203/2413] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:45:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:45:26.286091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:26.287605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:26.299747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6509d25b-1c6b-4040-9e49-0d7c993c4897 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:26.304699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:26.305552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.305835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.306117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] PATH_INFO: `/v3/projects/dbcd277f142a4a328f7553029beb300e` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.307312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbcd277f142a4a328f7553029beb300e)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:26.309258 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:26.405095 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbcd277f142a4a328f7553029beb300e', 'request_id': 'req-7029bfc5-433a-4f99-82d3-687648125817'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:26.461291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbcd277f142a4a328f7553029beb300e', 'request_id': 'req-7029bfc5-433a-4f99-82d3-687648125817'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:26.467126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbcd277f142a4a328f7553029beb300e', 'request_id': 'req-7029bfc5-433a-4f99-82d3-687648125817'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:26.478070 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] The token cache is being invalidate because project dbcd277f142a4a328f7553029beb300e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:45:26.478350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:26.479008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7029bfc5-433a-4f99-82d3-687648125817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:45:26.480644 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1211/2414] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:45:26 2022] DELETE /identity/v3/projects/dbcd277f142a4a328f7553029beb300e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:45:26.486015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.486302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.486578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.504136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:26.505304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:26.562458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1204/2415] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:26.572147 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.572555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.572841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.591843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:26.593040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:26.600848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1212/2416] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:26.605145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:26.614821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3348265e-3ba7-4e2d-8844-5bdcfb8afea3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:26.621813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:26.622972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.623488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.624055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.625495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:45:26.626557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:26.629643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:26.642995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1205/2417] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:45:26.650709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:26.659933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be491ef-8d37-4862-9389-d450e8137091 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:26.664515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:26.665183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.665482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.665757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.666589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:26.668590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:26.794700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1213/2418] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:45:26 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:45:26.799551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:26.809384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0e3851-680c-4e49-8800-62c6ce03b2be admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:26.858734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:26.865905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:26.866641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.866968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.867243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.867982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:26.869913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:26.963761 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1206/2419] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:45:26 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:45:26.968032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:26.978286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d13faa-2dfc-4d41-be5c-92e30bbc2e79 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:26.982866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:26.983544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:26.983861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:26.984139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:26.984888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:26.986998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:26.997388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1214/2420] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:45:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:45:27.001908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:27.011938 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb79c7-7cf9-4436-8c1d-ae98dfee64a0 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:27.016830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542adf6-f5a7-47c6-b536-e035697c9ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:27.017478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542adf6-f5a7-47c6-b536-e035697c9ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:27.017767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542adf6-f5a7-47c6-b536-e035697c9ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:27.018047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542adf6-f5a7-47c6-b536-e035697c9ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:27.018764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542adf6-f5a7-47c6-b536-e035697c9ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:27.020403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542adf6-f5a7-47c6-b536-e035697c9ba4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:27.031872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1207/2421] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:45:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:45:27.035957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:27.045756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed995b-ae9c-4799-8273-ae68d46e9383 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:27.050968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:27.051869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:27.052157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:27.052435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] PATH_INFO: `/v3/projects/64507ac6234948fba1aa5c64216b7a1e/users/e826d6287bd34267b9614b94b1cd41e2/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:27.067038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64507ac6234948fba1aa5c64216b7a1e, user_id=e826d6287bd34267b9614b94b1cd41e2, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:27.069723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:27.146951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1215/2422] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:45:27 2022] PUT /identity/v3/projects/64507ac6234948fba1aa5c64216b7a1e/users/e826d6287bd34267b9614b94b1cd41e2/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:45:27.152806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416d9f7-340a-4367-bc12-d31848caa897 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:27.153095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416d9f7-340a-4367-bc12-d31848caa897 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:27.153376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416d9f7-340a-4367-bc12-d31848caa897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:27.170838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6416d9f7-340a-4367-bc12-d31848caa897 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:27.172080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6416d9f7-340a-4367-bc12-d31848caa897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:27.232607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1208/2423] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:27.239825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:27.241782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:27.251835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:27.301146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7720642c-21c1-43fd-8049-b4c404f7d427 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:27.305139 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:27.305783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:27.306065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:27.306348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:27.316220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:27.317850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:27.327604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1216/2424] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:45:33.369780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:33.369780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:33.369780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:33.402935 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] MFA Rules not processed for user `e826d6287bd34267b9614b94b1cd41e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:33.404299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:33.475733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1209/2425] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:45:33.478559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:33.492240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898d7b-2333-4bc1-87e3-f300cb2d80b8 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:33.497107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:33.498055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:33.498229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:33.498518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:33.499219 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:33.501842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:33.601679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1217/2426] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:45:33 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:45:33.606060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:33.618536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da7d1eb-8ce0-4f2b-a056-5ff7fba48ddd None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:33.678717 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:33.679542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:33.679942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:33.680280 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:33.684404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:33.684404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:33.694251 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:33.764272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1210/2427] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:45:33 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:45:33.768696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:33.779097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0365c-f3fa-482a-9c62-0100a1887b0b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:33.783833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:33.784492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:33.784781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:33.785059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:33.785782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:33.787460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:33.798578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1218/2428] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:45:33.802823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:33.812801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90c215-5e8b-4707-9cd3-d894ed8b014c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:33.818283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:33.819224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:33.819507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:33.819832 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] PATH_INFO: `/v3/projects/bfdee22b8b8d405e87f78d7110aa5cd7/users/cd2e522377444450b995b36874f9101b/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:33.834322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfdee22b8b8d405e87f78d7110aa5cd7, user_id=cd2e522377444450b995b36874f9101b, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:33.837008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:33.933030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1211/2429] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:45:33 2022] PUT /identity/v3/projects/bfdee22b8b8d405e87f78d7110aa5cd7/users/cd2e522377444450b995b36874f9101b/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:45:33.937186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:33.947152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b75a3-e683-49db-bf6a-3f6ea6d3f2b6 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:34.005822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:34.006496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:34.006816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:34.007094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:34.007853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:34.009460 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:34.020376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1219/2430] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:45:33 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:45:34.025487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:34.042958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1df44-e0c4-471c-bacd-34644ffa1e17 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:34.048684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4df4c6-7231-4566-b0fb-8a5e4d658e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:34.049843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4df4c6-7231-4566-b0fb-8a5e4d658e01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:34.050135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4df4c6-7231-4566-b0fb-8a5e4d658e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:34.050414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4df4c6-7231-4566-b0fb-8a5e4d658e01 admin admin] PATH_INFO: `/v3/projects/bfdee22b8b8d405e87f78d7110aa5cd7/users/cd2e522377444450b995b36874f9101b/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:34.052518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4df4c6-7231-4566-b0fb-8a5e4d658e01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfdee22b8b8d405e87f78d7110aa5cd7, user_id=cd2e522377444450b995b36874f9101b, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:34.055241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4df4c6-7231-4566-b0fb-8a5e4d658e01 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:34.066960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1212/2431] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:45:34 2022] PUT /identity/v3/projects/bfdee22b8b8d405e87f78d7110aa5cd7/users/cd2e522377444450b995b36874f9101b/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:45:36.687503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:36.810497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc5740-65c1-43c2-9120-889bd6f94788 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:36.885238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:36.915913 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:36.920481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:36.920481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:36.920481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:36.920481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:37.007752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:37.009773 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:37.023733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1220/2432] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:45:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:45:40.552018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731499-f7cd-4190-8742-e9fd7d1e67cc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:40.552513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731499-f7cd-4190-8742-e9fd7d1e67cc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:40.552726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96731499-f7cd-4190-8742-e9fd7d1e67cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:40.588991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-96731499-f7cd-4190-8742-e9fd7d1e67cc None None] MFA Rules not processed for user `cd2e522377444450b995b36874f9101b`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:40.590203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-96731499-f7cd-4190-8742-e9fd7d1e67cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:40.709610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1213/2433] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:40.721290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:40.722873 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:40.901592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf7772-d9c0-47f5-a74e-da8afe48df28 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:40.906336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb84e744681149aea78fcc984605bf33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:40.907812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:40.908105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:40.908388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:41.005448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:41.007000 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:41.020330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1221/2434] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:45:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:45:41.262536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6503-bbda-46da-8dfe-94f985b97d39 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:41.262993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6503-bbda-46da-8dfe-94f985b97d39 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:41.263377 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6503-bbda-46da-8dfe-94f985b97d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:41.286345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-398c6503-bbda-46da-8dfe-94f985b97d39 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:41.287549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-398c6503-bbda-46da-8dfe-94f985b97d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:41.349192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1214/2435] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:45:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:41.370558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:41.372003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:41.388572 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:41.447790 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a90101f-e312-4c6a-bed4-2d568c837183 service glance] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:41.452870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f87c2-1c58-4e5b-b6d7-c71c80ea7409 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:41.453542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f87c2-1c58-4e5b-b6d7-c71c80ea7409 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:41.454503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f87c2-1c58-4e5b-b6d7-c71c80ea7409 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:41.454771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f87c2-1c58-4e5b-b6d7-c71c80ea7409 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:41.462866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f87c2-1c58-4e5b-b6d7-c71c80ea7409 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:41.464653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f87c2-1c58-4e5b-b6d7-c71c80ea7409 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:41.473386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1222/2436] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:45:51.341413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=3,keystone:SELECT=76 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:51.489976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=26 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:45:51.863534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df54b-8d5f-4a10-ad6a-eecddc57a2ef None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:51.863860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df54b-8d5f-4a10-ad6a-eecddc57a2ef None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:51.864145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df54b-8d5f-4a10-ad6a-eecddc57a2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:51.881888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-2a6df54b-8d5f-4a10-ad6a-eecddc57a2ef None None] MFA Rules not processed for user `a10ffb60631240fdb6b4a69d6d7f73e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:51.883079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2a6df54b-8d5f-4a10-ad6a-eecddc57a2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:51.944494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1215/2437] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:52.020894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:52.021184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:52.021481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:52.038704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] MFA Rules not processed for user `e826d6287bd34267b9614b94b1cd41e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:52.039935 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:52.046883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1223/2438] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:45:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:56.572352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee8fe-7c3a-49a8-bb6c-fb196f3825c3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:56.572862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee8fe-7c3a-49a8-bb6c-fb196f3825c3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:56.572979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ee8fe-7c3a-49a8-bb6c-fb196f3825c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:56.591840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-030ee8fe-7c3a-49a8-bb6c-fb196f3825c3 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:45:56.593065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-030ee8fe-7c3a-49a8-bb6c-fb196f3825c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:45:56.601192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1216/2439] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:45:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:45:56.617402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:45:56.630079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5990f4-eee6-4b66-bcab-ce150bf88a21 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:45:56.635611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:45:56.636376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:45:56.636905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:45:56.637251 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:45:56.646418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:45:56.648102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:45:56.656694 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1224/2440] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:45:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:46:06.586201 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=28 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:46:06.653728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=9 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:46:17.302387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f8cf7-a2b1-44fc-adf6-28cbc34162d7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:46:17.305856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f8cf7-a2b1-44fc-adf6-28cbc34162d7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:46:17.305856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f8cf7-a2b1-44fc-adf6-28cbc34162d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:46:17.320713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-624f8cf7-a2b1-44fc-adf6-28cbc34162d7 None None] MFA Rules not processed for user `3d3896bd70dd43fe95f7125f9f4693a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:46:17.324448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-624f8cf7-a2b1-44fc-adf6-28cbc34162d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:46:17.375745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1217/2441] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:46:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:46:27.369020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:46:29.005259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:46:29.014429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5518c17-1639-4775-baf3-2d6ed6e8a960 service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:46:29.018260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:46:29.018888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:46:29.019176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:46:29.019454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:46:29.028318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:46:29.029770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:46:29.039923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1225/2442] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:46:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:46:38.939729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:46:38.940249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:46:38.940349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:46:38.958167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] MFA Rules not processed for user `a47ab3ebbc9a4a429e5df2a616f305f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:46:38.959365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:46:39.012338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1218/2443] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:46:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:46:39.035570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=6 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:46:49.005821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:02.912267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:02.913831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:02.980130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc61ed1-13cd-4150-90f2-4fa868e1d90d service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:02.984698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:02.985556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:02.985840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:02.986115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] PATH_INFO: `/v3/users/a10ffb60631240fdb6b4a69d6d7f73e2` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:02.987839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a10ffb60631240fdb6b4a69d6d7f73e2)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:02.989957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:03.129625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a10ffb60631240fdb6b4a69d6d7f73e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.130197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a10ffb60631240fdb6b4a69d6d7f73e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.173869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a10ffb60631240fdb6b4a69d6d7f73e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.181371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a10ffb60631240fdb6b4a69d6d7f73e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.193281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-7eec1342-b886-4ae1-a8aa-250b591b6f8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a10ffb60631240fdb6b4a69d6d7f73e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.199852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1226/2444] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:02 2022] DELETE /identity/v3/users/a10ffb60631240fdb6b4a69d6d7f73e2 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:03.455588 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:03.469519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c89c2-7263-4630-ae75-30f7629ad7ab None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:03.527602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:03.528608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:03.528887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:03.529161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] PATH_INFO: `/v3/projects/ffb14a29d13a483ba8733e0bb3a56a17` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:03.530314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb14a29d13a483ba8733e0bb3a56a17)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:03.532290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:03.642310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb14a29d13a483ba8733e0bb3a56a17', 'request_id': 'req-9c3fa5d6-c87a-4817-ac2f-d807607e1944'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.688265 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb14a29d13a483ba8733e0bb3a56a17', 'request_id': 'req-9c3fa5d6-c87a-4817-ac2f-d807607e1944'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.693808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb14a29d13a483ba8733e0bb3a56a17', 'request_id': 'req-9c3fa5d6-c87a-4817-ac2f-d807607e1944'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.704535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] The token cache is being invalidate because project ffb14a29d13a483ba8733e0bb3a56a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:03.704821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.705446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9c3fa5d6-c87a-4817-ac2f-d807607e1944 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:03.707466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1219/2445] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:03 2022] DELETE /identity/v3/projects/ffb14a29d13a483ba8733e0bb3a56a17 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:03.712525 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:03.712863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:03.713146 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:03.731597 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:03.732813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:03.795459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1227/2446] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:03.803041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:03.804404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:03.804404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:03.822197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:03.823387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:03.831288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1220/2447] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:03.835971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:03.845424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b986780-24cf-4457-8996-1959d37023a1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:03.849994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:03.850646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:03.850912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:03.851184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:03.851933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:03.855968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:03.855968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:03.865611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1228/2448] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:03.869700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:03.879205 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bac1c-97e8-4139-9deb-9283f42803c6 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:03.884655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:03.884655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:03.884880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:03.884979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:03.885661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:03.887543 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:03.965965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1221/2449] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:03 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:03.968630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:03.978553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa699c3-03a6-4e7d-89bc-9a63bc6fc44f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:03.982286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:04.034417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:04.035108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:04.035424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:04.035734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:04.036427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:04.038334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:04.128272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1229/2450] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:03 2022] POST /identity/v3/users => generated 284 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:04.132868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:04.143507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f899e-4ed4-4a0c-8e9e-4125af6ebf55 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:04.148674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:04.149324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:04.149607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:04.149883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:04.150603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:04.152226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:04.162377 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1222/2451] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:04.166266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:04.176673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30e3585-9ebd-44fb-8d0e-0870931aaed9 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:04.181459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c953030-7986-4f40-a158-105441551049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:04.182102 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c953030-7986-4f40-a158-105441551049 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:04.182383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c953030-7986-4f40-a158-105441551049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:04.182678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c953030-7986-4f40-a158-105441551049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:04.183391 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c953030-7986-4f40-a158-105441551049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:04.185008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c953030-7986-4f40-a158-105441551049 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:04.195328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1230/2452] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:04.199172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:04.209025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabf6951-c2fe-4707-98a7-df3408c7ee78 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:04.213640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:04.214501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:04.214786 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:04.215063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] PATH_INFO: `/v3/projects/9eef37280d91495980e35bd4a636532d/users/1e488dd55c114506a636c2e953b19647/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:04.229443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eef37280d91495980e35bd4a636532d, user_id=1e488dd55c114506a636c2e953b19647, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:04.232141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:04.306349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1223/2453] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:04 2022] PUT /identity/v3/projects/9eef37280d91495980e35bd4a636532d/users/1e488dd55c114506a636c2e953b19647/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:04.323472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:04.323794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:04.324075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:04.349880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] MFA Rules not processed for user `1e488dd55c114506a636c2e953b19647`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:04.351128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:04.413452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1231/2454] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:04.424345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:04.426603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:04.457011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:04.482011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb9360-3169-461b-bfe6-e85c0820646b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:04.485671 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:04.486303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:04.486620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:04.486900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:04.496462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:04.497881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:04.507115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1224/2455] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:04 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:06.180401 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:06.181831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:06.194729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49326365-f4ff-4f13-a404-4893930b0f12 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:06.198372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:06.200697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:06.200697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:06.200697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:06.201372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:06.274478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:06.275971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:06.281801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:06.287473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1232/2456] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:06.777112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:06.778632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:06.841605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd457f1-f504-4e4f-9190-7304631d6d39 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:06.845197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:06.845837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:06.846116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:06.846388 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:06.848274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:06.862075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:06.863479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a33b11-df92-4883-a234-05c8e4f55594 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:06.874530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1225/2457] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:14.127193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:14.192008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b4335-9b95-4b9f-840e-0f55c7534a66 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:14.196555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:14.197420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:14.197763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:14.198055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] PATH_INFO: `/v3/users/1e488dd55c114506a636c2e953b19647` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:14.199225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e488dd55c114506a636c2e953b19647)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:14.201157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:14.359754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e488dd55c114506a636c2e953b19647'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:14.359754 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e488dd55c114506a636c2e953b19647'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:14.430759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e488dd55c114506a636c2e953b19647'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:14.438326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e488dd55c114506a636c2e953b19647'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:14.450541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e488dd55c114506a636c2e953b19647'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:14.457011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1233/2458] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:14 2022] DELETE /identity/v3/users/1e488dd55c114506a636c2e953b19647 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:14.461745 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:14.462217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:14.462332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:14.478855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:14.480079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:14.538296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1226/2459] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:47:14.547869 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:14.549441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:14.605998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016e332-8578-477e-b1d8-9caad935de86 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:14.609653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25db0d61-f147-48d2-9109-258ba569ecf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:14.610342 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0d61-f147-48d2-9109-258ba569ecf0 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:14.610575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0d61-f147-48d2-9109-258ba569ecf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:14.610854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25db0d61-f147-48d2-9109-258ba569ecf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:14.620238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0d61-f147-48d2-9109-258ba569ecf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:14.621662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25db0d61-f147-48d2-9109-258ba569ecf0 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:14.630990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1234/2460] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:15.061185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.062623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:15.074676 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb9542-8f38-492c-b12f-df0cc5cf3770 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.080218 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.081077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.081354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.081619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] PATH_INFO: `/v3/projects/9eef37280d91495980e35bd4a636532d` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.082788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eef37280d91495980e35bd4a636532d)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.084736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.191217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eef37280d91495980e35bd4a636532d', 'request_id': 'req-80573624-f1c1-4dc5-8a38-e1f688ed38a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:15.265911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eef37280d91495980e35bd4a636532d', 'request_id': 'req-80573624-f1c1-4dc5-8a38-e1f688ed38a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:15.271932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eef37280d91495980e35bd4a636532d', 'request_id': 'req-80573624-f1c1-4dc5-8a38-e1f688ed38a7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:15.273808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:15.282988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] The token cache is being invalidate because project 9eef37280d91495980e35bd4a636532d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:15.286379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:15.286379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-80573624-f1c1-4dc5-8a38-e1f688ed38a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:15.286379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1227/2461] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:15 2022] DELETE /identity/v3/projects/9eef37280d91495980e35bd4a636532d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:15.291784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.292235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.292343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.310049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:15.311269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:15.327946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:15.370337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1235/2462] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:15.377950 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.378457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.378816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.398025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:15.399221 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:15.408047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1228/2463] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:15.411339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.421801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d82f9c-8b06-4a09-8fed-e8b387c65cfb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.426693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.427355 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.428479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.428767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.429507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:15.430208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.432181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.444776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1236/2464] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:15.449445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.458785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b67d0-f88b-47c7-8f2b-cc0de3f54fa1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.464017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.464756 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.465103 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.465442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.466198 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.468177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.547231 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1229/2465] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:15 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:47:15.551156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.561157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501989b-ced8-443b-b615-4832374aab15 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.616904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.617562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.617847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.618127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.618814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.620796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.699740 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1237/2466] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:15 2022] POST /identity/v3/users => generated 285 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:15.703766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.714485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79768bc-b481-4676-acf1-27f77515e8d9 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.719068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.719743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.720033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.720313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.721086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.722698 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.733610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1230/2467] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:15.737615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.747530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30bbe-7da0-4c64-83bc-e4097a80acbc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.752561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98293348-fb00-429b-b64e-6965c82044cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.753218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98293348-fb00-429b-b64e-6965c82044cb admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.753504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98293348-fb00-429b-b64e-6965c82044cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.753785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98293348-fb00-429b-b64e-6965c82044cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.754541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98293348-fb00-429b-b64e-6965c82044cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.756226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98293348-fb00-429b-b64e-6965c82044cb admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.766879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1238/2468] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:47:15.771073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:15.781486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b52cad-228e-41a0-8d03-4c1861157118 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:15.786086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:15.786942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.787221 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.787495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] PATH_INFO: `/v3/projects/b21a0dd30cae4a0aa8669b8c3fea324e/users/8f0b6464bf164d56a8c8a47525249e73/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.801815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21a0dd30cae4a0aa8669b8c3fea324e, user_id=8f0b6464bf164d56a8c8a47525249e73, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:15.804512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:15.865384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1231/2469] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:15 2022] PUT /identity/v3/projects/b21a0dd30cae4a0aa8669b8c3fea324e/users/8f0b6464bf164d56a8c8a47525249e73/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:15.875570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:15.875936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:15.876205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:15.903464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] MFA Rules not processed for user `8f0b6464bf164d56a8c8a47525249e73`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:15.904736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:15.967752 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1239/2470] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:15 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:15.972290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.033492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f9fbb-e653-497c-b488-5b9e875c7f5a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.038114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.038763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.039048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.039321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.040058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.041964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.139130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1232/2471] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:15 2022] POST /identity/v3/projects => generated 369 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:16.140939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.151503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65173b8e-cdee-4937-8803-3aba230400a1 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.163927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:16.211333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.212092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.212361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.212638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.213454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.215277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.392295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1240/2472] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:16 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:16.396282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.406255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c9c1c-5d66-4db7-a4a6-4b85765169e4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.410715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.411354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.411634 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.411940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.412654 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.414227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.424401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1233/2473] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:16.428863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.438693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78091f9c-3627-4053-8baf-a5832412a965 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.443338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.444812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.445092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.445370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] PATH_INFO: `/v3/projects/9012611830c64b3b99528cb7d82695c5/users/9ef9b57b10b34377a375348f69502d4f/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.460312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9012611830c64b3b99528cb7d82695c5, user_id=9ef9b57b10b34377a375348f69502d4f, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.463098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.580731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1241/2474] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:16 2022] PUT /identity/v3/projects/9012611830c64b3b99528cb7d82695c5/users/9ef9b57b10b34377a375348f69502d4f/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:47:16.584408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.594525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6851b-f7bb-43b1-b67e-0c7e6c227d57 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.652976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe9b15-cb99-4cf9-b17b-aea059c7a4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.653585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe9b15-cb99-4cf9-b17b-aea059c7a4fa admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.653891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe9b15-cb99-4cf9-b17b-aea059c7a4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.654169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe9b15-cb99-4cf9-b17b-aea059c7a4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.654894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe9b15-cb99-4cf9-b17b-aea059c7a4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.656524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe9b15-cb99-4cf9-b17b-aea059c7a4fa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.670422 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1234/2475] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:16.672189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.692578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b257f6-3541-4681-b682-6010a7d4b341 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.697903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.699825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.700277 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.700625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] PATH_INFO: `/v3/projects/9012611830c64b3b99528cb7d82695c5/users/9ef9b57b10b34377a375348f69502d4f/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.702787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9012611830c64b3b99528cb7d82695c5, user_id=9ef9b57b10b34377a375348f69502d4f, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.705646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.726083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1242/2476] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:16 2022] PUT /identity/v3/projects/9012611830c64b3b99528cb7d82695c5/users/9ef9b57b10b34377a375348f69502d4f/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:16.743684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.744112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.744498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.750559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:16.771302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] MFA Rules not processed for user `9ef9b57b10b34377a375348f69502d4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:16.772558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:16.840991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1235/2477] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:16 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:16.852817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:16.854514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:16.910831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d917b-e95a-4787-b671-e2773344ae83 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:16.915336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356081d-e773-4136-b087-9dab590ffb7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:16.916043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356081d-e773-4136-b087-9dab590ffb7e service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:16.916327 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356081d-e773-4136-b087-9dab590ffb7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:16.916607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356081d-e773-4136-b087-9dab590ffb7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:16.925853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356081d-e773-4136-b087-9dab590ffb7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:16.927575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b356081d-e773-4136-b087-9dab590ffb7e service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:16.937372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1243/2478] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:19.629964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:19.690262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646634bb-a263-4d20-8b8b-17556f7b9549 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:19.694730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:19.695581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:19.695884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:19.696155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] PATH_INFO: `/v3/users/8f0b6464bf164d56a8c8a47525249e73` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:19.697520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f0b6464bf164d56a8c8a47525249e73)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:19.699466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:19.849379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f0b6464bf164d56a8c8a47525249e73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:19.849940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f0b6464bf164d56a8c8a47525249e73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:19.921851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f0b6464bf164d56a8c8a47525249e73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:19.933978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f0b6464bf164d56a8c8a47525249e73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:19.940758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f0b6464bf164d56a8c8a47525249e73'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:19.946700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1236/2479] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:19 2022] DELETE /identity/v3/users/8f0b6464bf164d56a8c8a47525249e73 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:19.951225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:19.951696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:19.951841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:19.969684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:19.970875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:20.027992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1244/2480] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:20.037415 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:20.039392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:20.049101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:20.095330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75fb9bd-490d-4165-9f60-247477ad96c8 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:20.099000 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:20.099691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:20.099947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:20.100835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:20.110269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:20.111709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:20.121806 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1237/2481] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:47:20.540984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:20.542411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:20.554262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af70f148-5658-4c4e-9cd8-2c80f175c839 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:20.558953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:20.559826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:20.560109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:20.560382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] PATH_INFO: `/v3/projects/b21a0dd30cae4a0aa8669b8c3fea324e` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:20.561648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21a0dd30cae4a0aa8669b8c3fea324e)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:20.563543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:20.573068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:20.665392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21a0dd30cae4a0aa8669b8c3fea324e', 'request_id': 'req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:20.699289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21a0dd30cae4a0aa8669b8c3fea324e', 'request_id': 'req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:20.704971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21a0dd30cae4a0aa8669b8c3fea324e', 'request_id': 'req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:20.715761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] The token cache is being invalidate because project b21a0dd30cae4a0aa8669b8c3fea324e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:20.730831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:20.730831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:20.730831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1245/2482] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:20 2022] DELETE /identity/v3/projects/b21a0dd30cae4a0aa8669b8c3fea324e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:20.731121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:20.731936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2e53c-486c-48d0-8338-3cb44db698d3 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:20.787401 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:20.788305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:20.788599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:20.788878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] PATH_INFO: `/v3/users/9ef9b57b10b34377a375348f69502d4f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:20.790021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ef9b57b10b34377a375348f69502d4f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:20.791948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:20.935477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ef9b57b10b34377a375348f69502d4f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:20.936052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ef9b57b10b34377a375348f69502d4f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.008298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ef9b57b10b34377a375348f69502d4f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.021177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ef9b57b10b34377a375348f69502d4f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.027994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0224ffea-d40a-47fe-b84d-097482b6c042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ef9b57b10b34377a375348f69502d4f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.033951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1238/2483] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:20 2022] DELETE /identity/v3/users/9ef9b57b10b34377a375348f69502d4f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:21.496910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:21.498543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:21.516706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0ee32-2aa3-47b1-8b3d-8874b45ee7d2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:21.576961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:21.577815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:21.578099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:21.578370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] PATH_INFO: `/v3/projects/9012611830c64b3b99528cb7d82695c5` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:21.579605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9012611830c64b3b99528cb7d82695c5)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:21.581809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:21.689255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9012611830c64b3b99528cb7d82695c5', 'request_id': 'req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.785198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9012611830c64b3b99528cb7d82695c5', 'request_id': 'req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.791819 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9012611830c64b3b99528cb7d82695c5', 'request_id': 'req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.802758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] The token cache is being invalidate because project 9012611830c64b3b99528cb7d82695c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:21.803055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.804232 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-d82177a2-52b1-4cfd-bcbc-f551499b3d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:21.805750 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1246/2484] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:21 2022] DELETE /identity/v3/projects/9012611830c64b3b99528cb7d82695c5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:21.810744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:21.811295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:21.811295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:21.829682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:21.830880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:21.856918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:21.891927 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1239/2485] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:21.899462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:21.899782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:21.900065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:21.919172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:21.920524 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:21.929204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1247/2486] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:21.933533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:21.943350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08f484b-8e4b-468f-93d6-f4614b6fb52e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:21.948846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:21.949490 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:21.949776 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:21.950052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:21.950779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:21.951085 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:21.952809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:21.964856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1240/2487] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:21.969119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:21.981596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7267e039-7f22-4937-b2c1-7b49b3347446 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:21.985827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:21.986482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:21.986770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:21.987046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:21.988014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:21.992112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.071335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:22.078370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1248/2488] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:21 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:22.082488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.092940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb957ab-60c1-4c2a-8297-12145ba20156 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.148649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.149301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.149586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.149862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.150549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.152482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.213945 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1241/2489] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:22 2022] POST /identity/v3/users => generated 290 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:22.217956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.229038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d00c7-f301-419f-8de1-72a58a35cb5c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.233763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.234430 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.234714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.234991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.235890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.237480 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.248215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1249/2490] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:47:22.252158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.262535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d646b9-12cd-4f6e-a2a0-656a09687445 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.267411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1634c1-2c43-4104-9f84-12701f20c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.268176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1634c1-2c43-4104-9f84-12701f20c10b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.268364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1634c1-2c43-4104-9f84-12701f20c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.268643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1634c1-2c43-4104-9f84-12701f20c10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.269441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1634c1-2c43-4104-9f84-12701f20c10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.270942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1634c1-2c43-4104-9f84-12701f20c10b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.281260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1242/2491] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:22.285375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.296119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bbb11-1893-43ce-827e-74abfbdb56d5 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.300747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.301606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.301881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.302150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] PATH_INFO: `/v3/projects/f9c209c0d69e434aaf7fe601196a065c/users/9d67ab2c70f24f339399afebccf875ca/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.316343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c209c0d69e434aaf7fe601196a065c, user_id=9d67ab2c70f24f339399afebccf875ca, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.319074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.398505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1250/2492] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:22 2022] PUT /identity/v3/projects/f9c209c0d69e434aaf7fe601196a065c/users/9d67ab2c70f24f339399afebccf875ca/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:22.410434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.410723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.411002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.437847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] MFA Rules not processed for user `9d67ab2c70f24f339399afebccf875ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:22.439032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:22.506116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1243/2493] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:22.511768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.517966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.519380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:22.553919 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:22.588586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b331e2-f211-4d09-9e64-ac84d530af38 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.593759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.594447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.594731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.595005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.596130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.597822 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.609947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19c0ab1-515f-4bb0-8cca-d6ac88a61157 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.615953 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.616608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.616892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.617175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.619039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:22.678752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.680418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.691839 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1244/2494] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:22.699781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1251/2495] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:22 2022] POST /identity/v3/projects => generated 379 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:22.700569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.711638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723a657-0b03-4faf-929e-a8dfcb5d37a4 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.767504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.768197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.768479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.768753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.769437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.771349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.831467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1245/2496] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:22 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:47:22.836125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.847010 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf26f810-9e4b-440d-8803-44506adf7538 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.851952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.852618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.852902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.853176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.853898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.855487 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.867642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1252/2497] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:47:22.871928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.883512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e49403-9717-44ed-8a1a-ed950145e8d5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:22.888189 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:22.889064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:22.889344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:22.889619 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] PATH_INFO: `/v3/projects/2ab1651db9b747789eb63d930fda33cf/users/2f820c67f6414fc788d5d153114e262d/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:22.904474 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab1651db9b747789eb63d930fda33cf, user_id=2f820c67f6414fc788d5d153114e262d, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:22.907253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:22.962109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1246/2498] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:22 2022] PUT /identity/v3/projects/2ab1651db9b747789eb63d930fda33cf/users/2f820c67f6414fc788d5d153114e262d/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:22.966145 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:22.975994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd353-cd34-41e3-8c6c-cffe9ff09a82 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:23.031173 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4532bc-13a1-42fe-bf0f-a081f6eab820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:23.031852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4532bc-13a1-42fe-bf0f-a081f6eab820 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:23.032133 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4532bc-13a1-42fe-bf0f-a081f6eab820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:23.032403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4532bc-13a1-42fe-bf0f-a081f6eab820 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:23.033106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4532bc-13a1-42fe-bf0f-a081f6eab820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:23.034674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4532bc-13a1-42fe-bf0f-a081f6eab820 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:23.044117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1253/2499] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:23.048262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:23.058765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aea54a-9e28-446b-9b40-55d3d5252328 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:23.063320 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:23.064206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:23.064485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:23.064758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] PATH_INFO: `/v3/projects/2ab1651db9b747789eb63d930fda33cf/users/2f820c67f6414fc788d5d153114e262d/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:23.066680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab1651db9b747789eb63d930fda33cf, user_id=2f820c67f6414fc788d5d153114e262d, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:23.069915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:23.079760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1247/2500] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:23 2022] PUT /identity/v3/projects/2ab1651db9b747789eb63d930fda33cf/users/2f820c67f6414fc788d5d153114e262d/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:23.089577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:23.089866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:23.090140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:23.115153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] MFA Rules not processed for user `2f820c67f6414fc788d5d153114e262d`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:23.116946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:23.136725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:23.183494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1254/2501] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:23.196300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:23.198005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:23.212012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:23.260648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94030963-f64e-4578-afac-f3ecaf312055 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:23.264483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:23.265396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:23.265680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:23.265957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:23.275538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:23.276990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:23.286476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1248/2502] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:24.172642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:24.182456 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d71135-2ec0-43f7-aa81-1e622d03f1a4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:24.187079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f820c67f6414fc788d5d153114e262d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ab1651db9b747789eb63d930fda33cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:24.187824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:24.188116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:24.188398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:24.189138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:24.189509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:24.191051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:24.202427 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1255/2503] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:24.206586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:24.215697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2125f0-1d40-4ed9-a051-960d0858c7e0 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:24.220451 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f820c67f6414fc788d5d153114e262d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ab1651db9b747789eb63d930fda33cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:24.221125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:24.224379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:24.224379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:24.224379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:24.224379 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:24.300850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1249/2504] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:24 2022] POST /identity/v3/projects => generated 354 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:24.305092 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:24.315030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fab05-d105-4757-8d8a-8410b8ddbb00 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:24.370690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f820c67f6414fc788d5d153114e262d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ab1651db9b747789eb63d930fda33cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:24.371336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:24.371622 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:24.372003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:24.372760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:24.373017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:24.374594 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:24.386193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1256/2505] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:24.421504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:24.432044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55779ba-04fd-49e0-944c-1419d8d8b6b7 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:24.436535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2f820c67f6414fc788d5d153114e262d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ab1651db9b747789eb63d930fda33cf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:24.437406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:24.437691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:24.437966 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] PATH_INFO: `/v3/projects/f9788eba43c14d67ae3e6e8b2191f458` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:24.439125 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f9788eba43c14d67ae3e6e8b2191f458)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:24.441047 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:24.538719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9788eba43c14d67ae3e6e8b2191f458', 'request_id': 'req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:24.590753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9788eba43c14d67ae3e6e8b2191f458', 'request_id': 'req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:24.596383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9788eba43c14d67ae3e6e8b2191f458', 'request_id': 'req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:24.607972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] The token cache is being invalidate because project f9788eba43c14d67ae3e6e8b2191f458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:24.608256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:24.608901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-9e8bdfeb-60a7-41d1-915f-c4851a38c4f4 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:24.610434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1250/2506] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:24 2022] DELETE /identity/v3/projects/f9788eba43c14d67ae3e6e8b2191f458 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:25.091616 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:25.093384 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:25.160693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2184844-95b2-419b-90b6-0886c39ed0c2 tempest-VolumeQuotaClassesTest-530826094 tempest-VolumeQuotaClassesTest-530826094-project-admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:25.203589 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:25.216447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:25.217259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:25.217541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:25.217813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] PATH_INFO: `/v3/users/9d67ab2c70f24f339399afebccf875ca` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:25.218979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d67ab2c70f24f339399afebccf875ca)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:25.220907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:25.353003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d67ab2c70f24f339399afebccf875ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:25.353654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d67ab2c70f24f339399afebccf875ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:25.407862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d67ab2c70f24f339399afebccf875ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:25.415583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d67ab2c70f24f339399afebccf875ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:25.427126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d67ab2c70f24f339399afebccf875ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:25.433245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1257/2507] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:25 2022] DELETE /identity/v3/users/9d67ab2c70f24f339399afebccf875ca => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:25.437707 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:25.438185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:25.438420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:25.454740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:25.455972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:25.515430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1251/2508] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:25.525550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:25.526967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:25.583239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba96fbe-d882-4b22-9531-41a5df933d1b admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:25.587506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:25.588178 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:25.588461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:25.588738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:25.597665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:25.599073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:25.608611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1258/2509] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:25.992864 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:25.994441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:26.010094 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5bade-99ed-488b-b8ac-cd1df0c13054 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:26.015036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:26.015931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:26.016196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:26.016471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] PATH_INFO: `/v3/projects/f9c209c0d69e434aaf7fe601196a065c` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:26.017754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c209c0d69e434aaf7fe601196a065c)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:26.019538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:26.031177 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:26.145286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c209c0d69e434aaf7fe601196a065c', 'request_id': 'req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.207990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c209c0d69e434aaf7fe601196a065c', 'request_id': 'req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.218185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c209c0d69e434aaf7fe601196a065c', 'request_id': 'req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.233425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] The token cache is being invalidate because project f9c209c0d69e434aaf7fe601196a065c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:26.233713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.234448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.236693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1252/2510] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:25 2022] DELETE /identity/v3/projects/f9c209c0d69e434aaf7fe601196a065c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:26.240707 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:26.251215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584415f3-65f4-44dd-b5b0-fc565b3a4cef service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:26.309363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:26.310290 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:26.310570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:26.310844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] PATH_INFO: `/v3/users/2f820c67f6414fc788d5d153114e262d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:26.312058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f820c67f6414fc788d5d153114e262d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:26.314009 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:26.324871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:26.484163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f820c67f6414fc788d5d153114e262d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.484810 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f820c67f6414fc788d5d153114e262d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.544927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f820c67f6414fc788d5d153114e262d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.551938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f820c67f6414fc788d5d153114e262d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.564571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-02a29f8e-d005-4110-9bc8-36d1068266aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f820c67f6414fc788d5d153114e262d'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:26.571005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1259/2511] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:26 2022] DELETE /identity/v3/users/2f820c67f6414fc788d5d153114e262d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:47:26.981568 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:26.983524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:26.996214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a00b2-ba24-48d4-9d6f-40b7d91000a3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.055740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.056597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.056882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.057160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] PATH_INFO: `/v3/projects/2ab1651db9b747789eb63d930fda33cf` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.058372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab1651db9b747789eb63d930fda33cf)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.060314 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.190334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab1651db9b747789eb63d930fda33cf', 'request_id': 'req-79a85216-8add-4c9a-a704-ca14efe7589c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:27.270576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab1651db9b747789eb63d930fda33cf', 'request_id': 'req-79a85216-8add-4c9a-a704-ca14efe7589c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:27.277028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab1651db9b747789eb63d930fda33cf', 'request_id': 'req-79a85216-8add-4c9a-a704-ca14efe7589c'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:27.288129 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] The token cache is being invalidate because project 2ab1651db9b747789eb63d930fda33cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:27.288409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:27.289235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-79a85216-8add-4c9a-a704-ca14efe7589c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:27.290759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1253/2512] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:26 2022] DELETE /identity/v3/projects/2ab1651db9b747789eb63d930fda33cf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:27.295780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.296249 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.296346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.313998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:27.315775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:27.375489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1260/2513] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:27.384976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.385185 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.385462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.403882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:27.405298 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:27.413227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1254/2514] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:27.417490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:27.426970 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1439ef-a847-48e9-9d02-25a2aa71921c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.432035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.432685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.432964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.433235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.433950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:27.434250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.435882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.448712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1261/2515] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:27.453049 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:27.462241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1940d92-62d8-4b85-a98b-d7bbc353778f None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.467056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.467816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.468100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.468387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.469079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.471206 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.598132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1255/2516] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:27 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:27.602165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:27.612061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04732a-476f-403f-ac0f-b27296078e5a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.666547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.667197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.667478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.667799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.668450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.670375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.784748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1262/2517] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:27 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:27.789278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:27.794642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:27.799837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d711d-153b-440f-b684-f1e6a344b2cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.805019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.805663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.805943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.806219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.806930 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.808537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.818492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1256/2518] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:27.826296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:27.832496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:27.836198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079a4de4-7e58-4299-83d1-1b1fcc475e76 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.840748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359c895-9939-478f-83a1-658446b2e9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.841373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359c895-9939-478f-83a1-658446b2e9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.841654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359c895-9939-478f-83a1-658446b2e9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.841926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359c895-9939-478f-83a1-658446b2e9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.842670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359c895-9939-478f-83a1-658446b2e9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.844276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359c895-9939-478f-83a1-658446b2e9d0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.854919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1263/2519] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:27.858960 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:27.869367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8527bf-3b66-4f31-8102-f72ee4844062 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:27.875441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:27.875441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.875441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.875441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] PATH_INFO: `/v3/projects/24390f124f1842d3959caf757f3d4399/users/7f2409506f4e4d23b2ca57c491d6f74a/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:27.890633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24390f124f1842d3959caf757f3d4399, user_id=7f2409506f4e4d23b2ca57c491d6f74a, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:27.893408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:27.979769 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1257/2520] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:27 2022] PUT /identity/v3/projects/24390f124f1842d3959caf757f3d4399/users/7f2409506f4e4d23b2ca57c491d6f74a/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:27.991975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:27.992263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:27.992555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.018241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] MFA Rules not processed for user `7f2409506f4e4d23b2ca57c491d6f74a`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:28.019450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:28.082225 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1264/2521] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:28.087475 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.147815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ca5a3-899d-4dd5-9b16-20d28b90f251 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.152427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.153069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.153350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.153626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.154310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.156259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.282057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1258/2522] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:28 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:28.283407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.294742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55121767-5d39-45a1-90a6-cfd39bfa4371 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.350018 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.351196 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.351482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.351800 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.352509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.354425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.420099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1265/2523] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:28 2022] POST /identity/v3/users => generated 299 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:28.424048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.435121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b002b0-81e6-4e9d-8019-0603d170677d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.439709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.440360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.440642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.440917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.441640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.443246 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.454456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1259/2524] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:28.462772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.472963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa200c7c-14a8-40b2-bed3-08e963eff29c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.478305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.479179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.479498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.479814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] PATH_INFO: `/v3/projects/63be098d02ee450d8524e91d2d54d989/users/369c4424ba6448f9ae29d83994fcfe39/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.494481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63be098d02ee450d8524e91d2d54d989, user_id=369c4424ba6448f9ae29d83994fcfe39, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.497185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.562975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1266/2525] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:28 2022] PUT /identity/v3/projects/63be098d02ee450d8524e91d2d54d989/users/369c4424ba6448f9ae29d83994fcfe39/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:28.566896 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.577662 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6559903-ee06-42c7-9c59-be38f683c366 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.635373 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49f1b5-2bfc-4adc-8ce7-23902657a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.636003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49f1b5-2bfc-4adc-8ce7-23902657a4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.636288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49f1b5-2bfc-4adc-8ce7-23902657a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.636863 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49f1b5-2bfc-4adc-8ce7-23902657a4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.637509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49f1b5-2bfc-4adc-8ce7-23902657a4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.639112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49f1b5-2bfc-4adc-8ce7-23902657a4e3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.650180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1260/2526] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:28.654291 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.664699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b69315-edc9-48b7-b743-296fc8e1e8b2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.669605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.670627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.670977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.671331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] PATH_INFO: `/v3/projects/63be098d02ee450d8524e91d2d54d989/users/369c4424ba6448f9ae29d83994fcfe39/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.673494 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63be098d02ee450d8524e91d2d54d989, user_id=369c4424ba6448f9ae29d83994fcfe39, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.676248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.686724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1267/2527] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:28 2022] PUT /identity/v3/projects/63be098d02ee450d8524e91d2d54d989/users/369c4424ba6448f9ae29d83994fcfe39/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:28.697309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.697609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.697894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.725486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] MFA Rules not processed for user `369c4424ba6448f9ae29d83994fcfe39`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:28.725486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:28.750328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:28.791504 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1261/2528] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:28.801929 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:28.803646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:28.843118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:28.860281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946c2f0-44a3-4a07-b668-3669accbfff7 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:28.863903 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:28.864540 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:28.864825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:28.865106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:28.920904 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:28.922353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:28.933238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1268/2529] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:29.827535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:29.829349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:29.902837 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e74363-b71f-4385-b776-1fcdf1433014 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:29.908365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:29.909008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:29.909294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:29.909576 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:29.911341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:29.967862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:29.969435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2e1f5-d952-4711-ae42-8af0e939dd53 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:29.979232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1262/2530] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:33.931078 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:33.939872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bfccf-a46f-4200-833f-fb8083db80c6 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:33.943426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:33.944611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:33.944887 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:33.945162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:33.953835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:33.955241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:33.965066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1269/2531] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:36.788538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:36.789593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:36.789714 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:36.807710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:36.808908 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:36.866095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1263/2532] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:47:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:36.912305 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:36.913719 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:36.979867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611e8fae-66d2-4b98-bb6a-aea6332f6f95 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:36.985015 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678056b-7aee-4e8c-b13a-dacf8d5a22ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:36.985660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678056b-7aee-4e8c-b13a-dacf8d5a22ae service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:36.985969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678056b-7aee-4e8c-b13a-dacf8d5a22ae service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:36.986244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678056b-7aee-4e8c-b13a-dacf8d5a22ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:36.996241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678056b-7aee-4e8c-b13a-dacf8d5a22ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:36.997961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678056b-7aee-4e8c-b13a-dacf8d5a22ae service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:37.006036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1270/2533] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:47:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:47:39.753740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:39.759239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:39.836110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ed811-b800-4566-943a-624ea4ef3f1b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:39.836110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:39.836110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:39.836110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:39.836587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] PATH_INFO: `/v3/users/7f2409506f4e4d23b2ca57c491d6f74a` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:39.836587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2409506f4e4d23b2ca57c491d6f74a)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:39.836587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:40.149437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2409506f4e4d23b2ca57c491d6f74a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:40.150411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2409506f4e4d23b2ca57c491d6f74a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:40.225249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2409506f4e4d23b2ca57c491d6f74a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:40.247194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2409506f4e4d23b2ca57c491d6f74a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:40.255260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2409506f4e4d23b2ca57c491d6f74a'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:40.261823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1264/2534] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:39 2022] DELETE /identity/v3/users/7f2409506f4e4d23b2ca57c491d6f74a => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:40.270881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:40.270881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:40.270881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:40.286962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:40.291941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:40.373228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1271/2535] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:40.382778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:40.384491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:40.490377 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae0773-c0ee-457c-9511-3eb9b3b2902b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:40.499799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:40.502106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:40.502106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:40.502106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:40.515897 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:40.517454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:40.537565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1265/2536] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:41.053158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:41.055390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:41.072206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c1170-4410-4093-b26a-6e4216d745b5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:41.077547 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:41.078424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:41.078710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:41.078990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] PATH_INFO: `/v3/projects/24390f124f1842d3959caf757f3d4399` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:41.080292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24390f124f1842d3959caf757f3d4399)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:41.082710 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:41.089331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:41.196793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24390f124f1842d3959caf757f3d4399', 'request_id': 'req-885d9695-db5f-429c-9544-2b2932730316'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.288309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24390f124f1842d3959caf757f3d4399', 'request_id': 'req-885d9695-db5f-429c-9544-2b2932730316'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.296581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24390f124f1842d3959caf757f3d4399', 'request_id': 'req-885d9695-db5f-429c-9544-2b2932730316'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.309254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] The token cache is being invalidate because project 24390f124f1842d3959caf757f3d4399 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:41.309916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.310762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.312555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1272/2537] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:41 2022] DELETE /identity/v3/projects/24390f124f1842d3959caf757f3d4399 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:41.316758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:41.329048 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e8fee-777a-443e-b9f2-c376b2388a4e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:41.367516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:41.407304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:41.408336 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:41.408688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:41.409021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] PATH_INFO: `/v3/users/369c4424ba6448f9ae29d83994fcfe39` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:41.414113 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=369c4424ba6448f9ae29d83994fcfe39)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:41.416513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:41.651566 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '369c4424ba6448f9ae29d83994fcfe39'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.653352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '369c4424ba6448f9ae29d83994fcfe39'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.739233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '369c4424ba6448f9ae29d83994fcfe39'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.761680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '369c4424ba6448f9ae29d83994fcfe39'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.780729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-c2aed8a5-cc99-4a24-b9c6-0cc6edc69b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '369c4424ba6448f9ae29d83994fcfe39'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:41.791825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1266/2538] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:41 2022] DELETE /identity/v3/users/369c4424ba6448f9ae29d83994fcfe39 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:42.273991 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:42.277309 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:42.297346 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885d9695-db5f-429c-9544-2b2932730316 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:42.396221 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:42.404718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:42.405283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:42.405745 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] PATH_INFO: `/v3/projects/63be098d02ee450d8524e91d2d54d989` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:42.407393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63be098d02ee450d8524e91d2d54d989)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:42.409245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:42.528068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63be098d02ee450d8524e91d2d54d989', 'request_id': 'req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:42.625027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63be098d02ee450d8524e91d2d54d989', 'request_id': 'req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:42.632167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63be098d02ee450d8524e91d2d54d989', 'request_id': 'req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:42.647484 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] The token cache is being invalidate because project 63be098d02ee450d8524e91d2d54d989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:42.648621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:42.649568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-9c70f5c3-50d5-46e3-9e85-ecec24571ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:42.651349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1273/2539] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:42 2022] DELETE /identity/v3/projects/63be098d02ee450d8524e91d2d54d989 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:42.660416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:42.660961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:42.664368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:42.683054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:42.684338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:42.755511 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1267/2540] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:42.764947 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:42.765236 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:42.765527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:42.786805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:42.788077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:42.796319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1274/2541] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:42.801130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:42.811513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4829fd49-9037-403e-a417-56e95b00d7b1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:42.816803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:42.817655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:42.817986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:42.818375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:42.819262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:42.819693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:42.822078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:42.844182 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1268/2542] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:47:42.844296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:42.854563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02efc-a1d2-4596-ac59-50de6f694b38 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:42.860125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:42.861086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:42.861813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:42.862245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:42.863252 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:42.865516 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:42.877954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:43.507896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1275/2543] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:42 2022] POST /identity/v3/projects => generated 359 bytes in 666 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:43.510875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:43.521484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5fc2a-6280-4cea-99b1-7e73d033ff44 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:43.555428 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:43.584940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:43.588559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:43.588559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:43.588559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:43.588559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:43.588975 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:43.677405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1269/2544] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:43 2022] POST /identity/v3/users => generated 280 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:43.683051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:43.695330 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b52065-d9d3-444c-83f2-524f1e6742e8 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:43.701030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:43.701702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:43.701984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:43.702454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:43.703362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:43.705286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:43.718265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1276/2545] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:43.722531 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:43.734021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7be5b7-1a6f-40f6-87e0-2e8f7b8e5b46 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:43.739610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33781112-c11b-4225-b21c-bc77ca90e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:43.740322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33781112-c11b-4225-b21c-bc77ca90e48f admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:43.740764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33781112-c11b-4225-b21c-bc77ca90e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:43.741044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33781112-c11b-4225-b21c-bc77ca90e48f admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:43.741861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33781112-c11b-4225-b21c-bc77ca90e48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:43.744510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33781112-c11b-4225-b21c-bc77ca90e48f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:43.756880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1270/2546] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:47:43.762073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:43.775366 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accf8842-594a-4327-ac7c-b504f03252e8 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:43.781403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:43.781403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:43.781403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:43.781834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] PATH_INFO: `/v3/projects/9ba1e7d560aa41118b466549bfd3cd84/users/e335850865de4424a20032f92b8af061/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:43.800978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba1e7d560aa41118b466549bfd3cd84, user_id=e335850865de4424a20032f92b8af061, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:43.805483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:43.912959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1277/2547] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:43 2022] PUT /identity/v3/projects/9ba1e7d560aa41118b466549bfd3cd84/users/e335850865de4424a20032f92b8af061/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:43.914980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:43.917580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:43.917580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:43.952744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] MFA Rules not processed for user `e335850865de4424a20032f92b8af061`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:43.954259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:44.045809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1271/2548] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:43 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:47:44.084388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:44.086823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:44.161969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2192cdab-76fd-4620-894a-a563dfba6375 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:44.172441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067480-631d-43ef-9af5-3f784e497e53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:44.173111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067480-631d-43ef-9af5-3f784e497e53 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:44.173398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067480-631d-43ef-9af5-3f784e497e53 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:44.173670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067480-631d-43ef-9af5-3f784e497e53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:44.190360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067480-631d-43ef-9af5-3f784e497e53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:44.194529 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067480-631d-43ef-9af5-3f784e497e53 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:44.219217 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1278/2549] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:44 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:44.222941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:44.314249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a399ab4c-fa59-481a-bfd6-182f647434c7 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:44.319020 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:44.320727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:44.321108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:44.321478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] PATH_INFO: `/v3/users/e335850865de4424a20032f92b8af061` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:44.322789 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e335850865de4424a20032f92b8af061)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:44.324834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:44.549120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:44.551866 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e335850865de4424a20032f92b8af061'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:44.552664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e335850865de4424a20032f92b8af061'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:44.659606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e335850865de4424a20032f92b8af061'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:44.674384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e335850865de4424a20032f92b8af061'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:44.683880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e335850865de4424a20032f92b8af061'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:44.696218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:44.697137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:44.697676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:44.699686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1272/2550] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:47:44 2022] DELETE /identity/v3/users/e335850865de4424a20032f92b8af061 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:44.719507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:44.721111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:44.786953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1279/2551] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:47:44.796323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:44.797813 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:44.861663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca3aaa-6ec2-4f64-9a81-0e4283396758 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:44.868767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03bd97-7d24-4060-8e6f-9d5e865d0443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:44.868767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03bd97-7d24-4060-8e6f-9d5e865d0443 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:44.868767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03bd97-7d24-4060-8e6f-9d5e865d0443 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:44.868767 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03bd97-7d24-4060-8e6f-9d5e865d0443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:44.877426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03bd97-7d24-4060-8e6f-9d5e865d0443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:44.878878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03bd97-7d24-4060-8e6f-9d5e865d0443 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:44.890814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1273/2552] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:45.348234 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:45.349831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:45.363765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82974e4-9831-4ea7-9487-fdd50e924d8b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:45.392618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:45.395123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:45.395420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:45.395732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] PATH_INFO: `/v3/projects/9ba1e7d560aa41118b466549bfd3cd84` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:45.397077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba1e7d560aa41118b466549bfd3cd84)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:45.405500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:45.536959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba1e7d560aa41118b466549bfd3cd84', 'request_id': 'req-76d592f0-3c2b-40b5-a9c1-48f894037ef9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:45.542452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:45.630744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba1e7d560aa41118b466549bfd3cd84', 'request_id': 'req-76d592f0-3c2b-40b5-a9c1-48f894037ef9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:45.637699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba1e7d560aa41118b466549bfd3cd84', 'request_id': 'req-76d592f0-3c2b-40b5-a9c1-48f894037ef9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:45.654916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] The token cache is being invalidate because project 9ba1e7d560aa41118b466549bfd3cd84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:47:45.655209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:45.655945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-76d592f0-3c2b-40b5-a9c1-48f894037ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:47:45.657871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1280/2553] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:47:45 2022] DELETE /identity/v3/projects/9ba1e7d560aa41118b466549bfd3cd84 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:47:45.662607 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:45.662937 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:45.663224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:45.684161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:45.685530 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:45.762844 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1274/2554] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:45.770732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:45.771266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:45.771378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:45.802042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:45.803261 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:45.811526 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1281/2555] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:45.817708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:45.830493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78239e-710a-42c7-8115-901d59146f02 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:45.836227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:45.839234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:45.839234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:45.839234 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:45.840895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:47:45.841774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:45.844345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:45.859327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1275/2556] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:47:45.861718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:45.873069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec161d-fcc4-4f8d-b325-aab615c7ce67 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:45.879112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:45.879999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:45.880353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:45.886160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:45.886160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:45.886160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:46.041688 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1282/2557] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:47:45 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:47:46.045883 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:46.057002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e18a1e-fdcd-4896-a8bb-aefeba811323 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:46.136664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:47:46.151539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:46.151539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:46.151539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:46.151539 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:46.151989 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:46.153660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:46.253713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:46.254285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1276/2558] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:47:46 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:47:46.266958 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3724e1-806c-48c2-a911-d91017c09d1c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:46.271957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:46.272655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:46.273192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:46.273482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:46.274243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:46.275959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:46.289518 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1283/2559] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:46.292098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:46.302747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1debf9c-73d1-4948-8bb9-f28100beeada admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:46.308596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c0aa8-3a79-4984-baf5-c8c2bca7b201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:46.309569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0aa8-3a79-4984-baf5-c8c2bca7b201 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:46.310260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0aa8-3a79-4984-baf5-c8c2bca7b201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:46.310912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c0aa8-3a79-4984-baf5-c8c2bca7b201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:46.315962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c0aa8-3a79-4984-baf5-c8c2bca7b201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:46.315962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c0aa8-3a79-4984-baf5-c8c2bca7b201 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:46.329623 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1277/2560] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:47:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:47:46.330799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:46.341713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23935d59-749f-4dbf-aed9-c8e325504550 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:46.346687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:46.347429 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:46.347751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:46.348034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] PATH_INFO: `/v3/projects/5d8cb37768214cc989b1162f01193885/users/91e0deef29b6400db557b2f69551c77f/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:46.364136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8cb37768214cc989b1162f01193885, user_id=91e0deef29b6400db557b2f69551c77f, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:46.367333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:47.053684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1284/2561] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:47:46 2022] PUT /identity/v3/projects/5d8cb37768214cc989b1162f01193885/users/91e0deef29b6400db557b2f69551c77f/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:47:47.065183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:47.065659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:47.065766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:47.094299 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] MFA Rules not processed for user `91e0deef29b6400db557b2f69551c77f`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:47:47.095527 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:47.191814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1278/2562] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:47:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:47:47.203836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:47.205685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:47.304031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9712da-7162-4424-985a-e30905bd81a2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:47.307821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:47.308505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:47.308933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:47.309229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:47.320870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:47.322306 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:47.343939 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1285/2563] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:49.358052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:49.359477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:49.420154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837545c4-7981-4b66-bb54-80c1232d6180 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:49.423771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:49.424417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:49.424703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:49.424980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:49.426803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:49.436585 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:49.438070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:49.446508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1279/2564] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:51.896419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:47:51.898452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:51.978621 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0a8f0-5d98-40f0-99aa-a5319992bb81 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:47:51.984419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:47:51.985081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:47:51.985371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:47:51.985658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:47:51.987610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:47:52.004364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:47:52.006126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:47:52.017975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1286/2565] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:47:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:47:59.442398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:01.213479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:01.221825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6349b5-3f68-4821-9320-65fbae701778 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:01.227716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:01.227716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:01.227716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:01.227716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:01.228985 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:01.241836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:01.243325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbcca06-a87b-468f-82e8-80e404f1836d service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:01.251811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1280/2566] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:01.785080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:01.786483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:01.795709 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:01.853884 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe22872-91f3-4266-8d15-3e9aaa77bb47 service nova] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:01.859019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:01.859674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:01.859971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:01.861522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:01.861987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:01.873611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:01.875179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e026df-a8c8-41a7-8558-2d20ce561f72 service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:01.885635 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1287/2567] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:04.085929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4c15-260f-4c14-96a5-82c1ea44bf19 None None] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.086553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4c15-260f-4c14-96a5-82c1ea44bf19 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.086553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df4c15-260f-4c14-96a5-82c1ea44bf19 None None] PATH_INFO: `/` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.087890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1281/2568] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:48:04.094440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2b32f-f687-423c-8835-7a3d17e0c051 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.094924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2b32f-f687-423c-8835-7a3d17e0c051 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.095032 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2b32f-f687-423c-8835-7a3d17e0c051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.113235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-85b2b32f-f687-423c-8835-7a3d17e0c051 None None] MFA Rules not processed for user `e527c9fc7ae1475595c311ac3215b51f`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:04.114491 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-85b2b32f-f687-423c-8835-7a3d17e0c051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:04.121908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1288/2569] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:48:04.138390 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4c15-260f-4c14-96a5-82c1ea44bf19 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:04.153810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df4c15-260f-4c14-96a5-82c1ea44bf19 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:04.158153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f9d58-76b7-4a6d-b123-d9a24f6cbecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:04.158827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9d58-76b7-4a6d-b123-d9a24f6cbecf service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.159098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9d58-76b7-4a6d-b123-d9a24f6cbecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.159378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f9d58-76b7-4a6d-b123-d9a24f6cbecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.172462 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f9d58-76b7-4a6d-b123-d9a24f6cbecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:04.173952 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f9d58-76b7-4a6d-b123-d9a24f6cbecf service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:04.183724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1282/2570] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:04.305353 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe21da-7f86-4663-b38c-b84b90a56e26 None None] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.305792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe21da-7f86-4663-b38c-b84b90a56e26 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.306298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbe21da-7f86-4663-b38c-b84b90a56e26 None None] PATH_INFO: `/` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.308829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1289/2571] 10.222.0.41 () {54 vars in 921 bytes} [Mon Jun 6 20:48:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 06 20:48:04.316267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9244-79d9-4cf8-bcee-cbaaec0b6c49 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.318057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9244-79d9-4cf8-bcee-cbaaec0b6c49 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.318057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb9244-79d9-4cf8-bcee-cbaaec0b6c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.342683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a7fb9244-79d9-4cf8-bcee-cbaaec0b6c49 None None] MFA Rules not processed for user `f310f6d84a3d42df9d473677ac5b6557`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:04.344442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a7fb9244-79d9-4cf8-bcee-cbaaec0b6c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:04.357986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1283/2572] 10.222.0.41 () {58 vars in 1018 bytes} [Mon Jun 6 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:48:04.370520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe21da-7f86-4663-b38c-b84b90a56e26 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:04.372335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3fbe21da-7f86-4663-b38c-b84b90a56e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:04.387223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbe21da-7f86-4663-b38c-b84b90a56e26 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:04.389772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.390065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.390348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.391757 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:04.392467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.392760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.393041 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.404104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:04.405459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:04.408361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:04.409563 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:04.417936 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1290/2573] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:04.473216 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1284/2574] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:48:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:48:04.486777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:04.488320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:04.572458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b6251-2196-4a5f-9cb3-70bea08b5d8b service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:04.577871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa113d-d706-4901-a805-da17331a0f55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:04.585798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa113d-d706-4901-a805-da17331a0f55 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:04.585798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa113d-d706-4901-a805-da17331a0f55 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:04.585798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa113d-d706-4901-a805-da17331a0f55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:04.590637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa113d-d706-4901-a805-da17331a0f55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:04.593025 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa113d-d706-4901-a805-da17331a0f55 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:04.610435 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1291/2575] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:48:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:48:08.812349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:08.929030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93475a7a-0fae-493c-8ba9-256685799d96 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:08.935775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:08.936751 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:08.937025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:08.937304 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] PATH_INFO: `/v3/users/91e0deef29b6400db557b2f69551c77f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:08.938533 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=91e0deef29b6400db557b2f69551c77f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:08.941188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:09.144230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91e0deef29b6400db557b2f69551c77f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:09.148278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91e0deef29b6400db557b2f69551c77f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:09.259493 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91e0deef29b6400db557b2f69551c77f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:09.275915 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91e0deef29b6400db557b2f69551c77f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:09.278534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:09.286748 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91e0deef29b6400db557b2f69551c77f'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:09.296443 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1285/2576] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:48:08 2022] DELETE /identity/v3/users/91e0deef29b6400db557b2f69551c77f => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:09.301205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:09.301871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:09.301871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:09.353898 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:09.355142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:09.527081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1292/2577] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:09.536843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:09.538356 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:09.749228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31528b52-05ab-4928-92ed-2732558c33ba admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:09.753408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:09.754245 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:09.754408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:09.754715 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:09.779550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:09.785969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:09.821970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1286/2578] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:10.143698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:10.145080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:10.161278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:10.405900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8606a4-ac2b-4eb9-ac5c-c13f46025746 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:10.432396 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:10.434628 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:10.435986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:10.438648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:10.444026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:10.480614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:10.494731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6a2c97-2edc-42b0-936c-02b8289bb43a service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:10.545891 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1293/2579] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:11.682172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:11.683722 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:11.698491 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987d5eba-725d-47f2-9d11-0dbd7ff2b91f service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:11.704496 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:11.705721 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:11.708228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:11.708228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] PATH_INFO: `/v3/projects/5d8cb37768214cc989b1162f01193885` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:11.715308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8cb37768214cc989b1162f01193885)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:11.724071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:11.950644 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8cb37768214cc989b1162f01193885', 'request_id': 'req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:12.042117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8cb37768214cc989b1162f01193885', 'request_id': 'req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:12.052071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8cb37768214cc989b1162f01193885', 'request_id': 'req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:12.102853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] The token cache is being invalidate because project 5d8cb37768214cc989b1162f01193885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:48:12.103162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:12.106683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1a45e7d4-7bd3-4cb0-b6a5-815e2fc04d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:12.111790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1287/2580] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:48:11 2022] DELETE /identity/v3/projects/5d8cb37768214cc989b1162f01193885 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:12.118069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:12.118658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:12.118658 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:12.157652 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:12.159060 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:12.338967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1294/2581] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:12.344514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:12.345002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:12.345478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:12.374402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:12.375648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:12.388610 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1288/2582] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:12.390240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:12.406609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359ce01-372a-4865-91ae-11ba47b4b907 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:12.431914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:12.432979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:12.433523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:12.435510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:12.436590 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:48:12.437138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:12.439462 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:12.477385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1295/2583] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:48:12.479853 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:12.494685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75229330-1a69-4f7d-bcf4-1989afa9fa5d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:12.510778 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:12.511592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:12.512365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:12.512757 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:12.513553 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:12.518876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:12.638943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1289/2584] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:48:12 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:48:12.643114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:12.655109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e739a15-853d-4949-beb7-ba4408afbe96 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:12.846562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:12.846562 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:12.846950 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:12.847544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:12.848824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:12.851558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:12.921450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:12.921450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:12.994302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:13.063354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b706-5b76-431d-b57d-159b786508e5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:13.100308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:13.101099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:13.101551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:13.101942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:13.109217 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:13.130003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:13.132100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:13.152998 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1290/2585] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:48:13.662108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1296/2586] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:48:12 2022] POST /identity/v3/users => generated 288 bytes in 1019 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:48:13.667329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:13.686036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716ec8b-17a8-484c-be02-675ed5b0a4f9 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:13.709142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:13.711845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:13.713151 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:13.713552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:13.719973 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:13.723586 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:13.773838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1291/2587] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:48:13.774051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:13.813909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b130798d-f118-43a6-9c02-774857f9a005 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:13.839085 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c9514-2f81-4957-98de-a90eda8cf501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:13.843724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c9514-2f81-4957-98de-a90eda8cf501 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:13.844793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c9514-2f81-4957-98de-a90eda8cf501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:13.845718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c9514-2f81-4957-98de-a90eda8cf501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:13.847901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c9514-2f81-4957-98de-a90eda8cf501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:13.849985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c9514-2f81-4957-98de-a90eda8cf501 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:13.855313 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:13.872144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1297/2588] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:48:13.877825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:13.917442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288a6e6-59ed-46e4-bf1f-6f47225d4702 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:13.954131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:13.957269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:13.961371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:13.962081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] PATH_INFO: `/v3/projects/c1ea53cfd795419cacebe7f2d071f37f/users/2bb3e3fc9a494cdf8abd264038af044c/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:14.019544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ea53cfd795419cacebe7f2d071f37f, user_id=2bb3e3fc9a494cdf8abd264038af044c, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:14.019544 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:14.134068 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1292/2589] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:48:13 2022] PUT /identity/v3/projects/c1ea53cfd795419cacebe7f2d071f37f/users/2bb3e3fc9a494cdf8abd264038af044c/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:14.139941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac1481-c80a-4fd5-ba22-85d8971f8c41 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:14.140340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac1481-c80a-4fd5-ba22-85d8971f8c41 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:14.140977 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac1481-c80a-4fd5-ba22-85d8971f8c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:14.204282 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-5bac1481-c80a-4fd5-ba22-85d8971f8c41 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:14.205924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5bac1481-c80a-4fd5-ba22-85d8971f8c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:14.351966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1298/2590] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:14.358418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:14.360528 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:14.456323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5abb2b1-c1fb-496e-b499-f9dd6e8af3bc admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:14.473403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:14.474043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:14.478203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:14.478203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:14.495209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:14.499346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:14.527050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1293/2591] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:22.135201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e261d1-ff96-495e-bdde-785ed96ac9dc None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.135851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e261d1-ff96-495e-bdde-785ed96ac9dc None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.135851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e261d1-ff96-495e-bdde-785ed96ac9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.167955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-c8e261d1-ff96-495e-bdde-785ed96ac9dc None None] MFA Rules not processed for user `2bb3e3fc9a494cdf8abd264038af044c`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:22.169208 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-c8e261d1-ff96-495e-bdde-785ed96ac9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:22.323530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1299/2592] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:48:22.333512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:22.338712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47d54c-cc23-4818-8ffc-8637e0b585a2 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:22.344471 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:22.345419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.345812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.346417 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.347467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:22.349594 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:22.413451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.413654 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.414555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.441338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:22.442702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:22.505083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1300/2593] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 20:48:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:48:22.518825 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:22.520407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:22.563233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1294/2594] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:48:22 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:48:22.568170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:22.604242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:22.675224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa730ec-8e61-4c21-aaa5-fd952916e460 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:22.677204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b478f637-2226-46a5-8d94-e4a4ea2dfd73 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:22.680809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:22.681538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.681828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.682160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.683173 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:22.687347 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:22.759567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1295/2595] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:48:22 2022] POST /identity/v3/users => generated 296 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:48:22.763328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:22.778836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:22.778836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.778836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.778836 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.785887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717c7a19-08d4-49e9-a47c-7b72feed2567 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:22.794886 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:22.795932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.796391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.796876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.797835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:22.801414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:22.803880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:22.816386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1296/2596] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:48:22.821403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:22.832518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582317ed-6fe7-4076-ab17-416ba496db11 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:22.838332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:22.839228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:22.839514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:22.839827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] PATH_INFO: `/v3/projects/27eabacaf0b14d5ba6f17560f8f4c762/users/610a04ce87fc4dc4b103811207caaa57/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:22.856181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eabacaf0b14d5ba6f17560f8f4c762, user_id=610a04ce87fc4dc4b103811207caaa57, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:22.858870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:22.877502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:22.879119 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:22.888459 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1301/2597] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:48:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 20:48:22.919014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1297/2598] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:48:22 2022] PUT /identity/v3/projects/27eabacaf0b14d5ba6f17560f8f4c762/users/610a04ce87fc4dc4b103811207caaa57/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:22.922830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:22.936664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3cdc28-5f22-4015-a8ed-f012c9689152 service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:23.029013 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:23.029821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:23.030216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:23.030605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:23.031734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:23.045536 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:23.046248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1302/2599] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:48:23.050827 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:23.068458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:23.070932 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd0ff6-c3de-4d74-a474-7bfc12490683 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:23.078153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad962d-a78f-4aaf-94a6-9b0f0140fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:23.079467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad962d-a78f-4aaf-94a6-9b0f0140fd2c admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:23.080044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad962d-a78f-4aaf-94a6-9b0f0140fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:23.086141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad962d-a78f-4aaf-94a6-9b0f0140fd2c admin admin] PATH_INFO: `/v3/projects/27eabacaf0b14d5ba6f17560f8f4c762/users/610a04ce87fc4dc4b103811207caaa57/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:23.086264 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:23.086365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad962d-a78f-4aaf-94a6-9b0f0140fd2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eabacaf0b14d5ba6f17560f8f4c762, user_id=610a04ce87fc4dc4b103811207caaa57, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:23.087986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad962d-a78f-4aaf-94a6-9b0f0140fd2c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:23.124136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1298/2600] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:48:23 2022] PUT /identity/v3/projects/27eabacaf0b14d5ba6f17560f8f4c762/users/610a04ce87fc4dc4b103811207caaa57/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:23.146827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32116d-5ca0-4f86-b596-0f6404d4a6da admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:23.204953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:23.207023 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:23.212725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:23.213157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:23.216955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:23.322478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:23.324136 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:23.339115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1303/2601] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:48:24.658825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:24.659821 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:24.660296 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:24.698838 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:24.708005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:24.818065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1299/2602] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 20:48:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:24.833682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:24.951960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b050a-6d5e-4893-b207-30a56b9f00e7 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:24.962422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:24.963184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:24.963481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:24.963883 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:24.981809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:24.984514 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:24.995724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1304/2603] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:48:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 06 20:48:29.889502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:29.890994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:30.031260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861fb11-bc43-497d-95b4-b3580ee8c8b5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:30.036874 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:30.037631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:30.037992 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:30.038227 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:30.043363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:30.057400 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:30.059026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b29fbba-6937-476b-b7f5-da94e8093977 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:30.083802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1300/2604] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:30.124124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:30.128457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:30.154419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8106d4-a308-43c4-b75d-2082dbe9ed95 service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:30.168202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:30.169479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:30.169938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:30.170365 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] PATH_INFO: `/v3/users/a47ab3ebbc9a4a429e5df2a616f305f0` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:30.179204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a47ab3ebbc9a4a429e5df2a616f305f0)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:30.179204 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:30.188711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:30.483376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a47ab3ebbc9a4a429e5df2a616f305f0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:30.487458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a47ab3ebbc9a4a429e5df2a616f305f0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:30.613437 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a47ab3ebbc9a4a429e5df2a616f305f0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:30.623625 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a47ab3ebbc9a4a429e5df2a616f305f0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:30.639974 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a47ab3ebbc9a4a429e5df2a616f305f0'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:30.652897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1305/2605] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:48:30 2022] DELETE /identity/v3/users/a47ab3ebbc9a4a429e5df2a616f305f0 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:30.953311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:30.954253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:30.954678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:30.988146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] MFA Rules not processed for user `610a04ce87fc4dc4b103811207caaa57`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:30.989996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:31.065967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:31.067527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:31.081940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:31.117357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1301/2606] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:31.131171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:31.134587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:31.260720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7037e-91b9-4f93-8590-0c5a396149d0 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:31.262919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dec70e-a6d1-4313-8e63-2336611526f0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:31.268309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:31.269081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:31.269431 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:31.270682 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:31.272934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:31.275555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:31.276697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:31.277227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:31.277717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] PATH_INFO: `/v3/projects/3a1af880b45d426aa752589183354a77` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:31.279209 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a1af880b45d426aa752589183354a77)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:31.284300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:31.403027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:31.404972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba18af4-448d-4079-8dac-749b98868038 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:31.418746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1302/2607] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:48:31.437239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a1af880b45d426aa752589183354a77', 'request_id': 'req-8eeb9be0-f412-426e-8dd8-756994b38eef'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:31.516110 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a1af880b45d426aa752589183354a77', 'request_id': 'req-8eeb9be0-f412-426e-8dd8-756994b38eef'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:31.524630 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a1af880b45d426aa752589183354a77', 'request_id': 'req-8eeb9be0-f412-426e-8dd8-756994b38eef'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:31.538174 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] The token cache is being invalidate because project 3a1af880b45d426aa752589183354a77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:48:31.539813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:31.540820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8eeb9be0-f412-426e-8dd8-756994b38eef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:31.546479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1306/2608] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:48:31 2022] DELETE /identity/v3/projects/3a1af880b45d426aa752589183354a77 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:48:31.547983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:31.548369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:31.549001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:31.583734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:31.585039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:31.751120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1303/2609] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:31.759696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:31.760003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:31.760294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:31.781005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:31.782343 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:31.795482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1307/2610] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:31.800392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:31.818110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c8f0e-3bc9-4ccc-8b87-fe140c49ac14 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:31.824061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:31.824818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:31.825108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:31.825389 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:31.826136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:48:31.826498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:31.828697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:31.876454 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1304/2611] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:48:31.881120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:31.888500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:31.893523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc63566-ddcf-43e0-af9d-2b490feb8053 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:31.904104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:31.904789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:31.905072 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:31.905349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:31.906065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:31.908030 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:32.072846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1308/2612] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:48:31 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:48:32.074301 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:32.081925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:32.088968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ecdddc-fdde-4fa9-b071-314c989a8d76 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:32.199175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:32.200083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:32.200469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:32.200846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:32.201657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:32.204152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:32.348279 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1305/2613] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:48:32 2022] POST /identity/v3/users => generated 292 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:48:32.349614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:32.360315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f505c-9821-4f18-b75d-fff3b4a5fdf2 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:32.365629 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:32.366229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:32.366910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:32.367201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:32.367987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:32.369643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:32.382751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1309/2614] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:48:32.386466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:32.401363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03478fc1-a150-49fe-86ae-9fda1a512eed admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:32.407495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade7ae20-2929-4088-83af-a936e04a84b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:32.408425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ae20-2929-4088-83af-a936e04a84b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:32.408835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ae20-2929-4088-83af-a936e04a84b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:32.409482 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade7ae20-2929-4088-83af-a936e04a84b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:32.410229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ae20-2929-4088-83af-a936e04a84b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:32.412046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade7ae20-2929-4088-83af-a936e04a84b4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:32.427038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1306/2615] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:48:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:48:32.431952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:32.446923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bee3ff-18e4-468f-88d2-2e574d7e6acc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:32.453118 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:32.454166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:32.454548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:32.454905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] PATH_INFO: `/v3/projects/19e92b8219814afd94d3d49bf9f13f30/users/7e7e03514f454d40aed80b81acab82ca/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:32.477852 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e92b8219814afd94d3d49bf9f13f30, user_id=7e7e03514f454d40aed80b81acab82ca, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:32.481879 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:32.557837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1310/2616] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:48:32 2022] PUT /identity/v3/projects/19e92b8219814afd94d3d49bf9f13f30/users/7e7e03514f454d40aed80b81acab82ca/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:32.563042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:32.563328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:32.563605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:32.627962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:32.630545 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:32.754640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1307/2617] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:32.771312 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:32.774162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:32.913049 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ceb3-1422-4a1f-8a08-82fba031dc4c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:32.918498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:32.919862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:32.920592 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:32.920961 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:32.942985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:32.950951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:32.966807 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1311/2618] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:38.193518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:38.198070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:38.389501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:38.392543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:38.444964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b12df0-d1b0-4815-b93f-bf5e2876cd1c service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:38.453373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:38.453923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c373bd-e9a3-478f-8b39-d8c268422faa None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:38.454831 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:38.455360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:38.455896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] PATH_INFO: `/v3/users/e826d6287bd34267b9614b94b1cd41e2` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:38.460692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e826d6287bd34267b9614b94b1cd41e2)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:38.470510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:38.483638 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:38.484350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:38.484636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:38.484913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:38.487213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:38.518466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:38.522410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:38.545013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1308/2619] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:38.786143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e826d6287bd34267b9614b94b1cd41e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:38.787766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e826d6287bd34267b9614b94b1cd41e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:38.886833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e826d6287bd34267b9614b94b1cd41e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:38.895317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e826d6287bd34267b9614b94b1cd41e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:38.912830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e826d6287bd34267b9614b94b1cd41e2'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:38.921655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1312/2620] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:48:38 2022] DELETE /identity/v3/users/e826d6287bd34267b9614b94b1cd41e2 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:39.314942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:39.343913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061ee05-fb8c-4742-9195-9c365c626689 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:39.361815 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:39.434795 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:39.436565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:39.437060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:39.437768 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] PATH_INFO: `/v3/projects/64507ac6234948fba1aa5c64216b7a1e` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:39.439053 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64507ac6234948fba1aa5c64216b7a1e)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:39.441176 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:39.590238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64507ac6234948fba1aa5c64216b7a1e', 'request_id': 'req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:39.678317 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64507ac6234948fba1aa5c64216b7a1e', 'request_id': 'req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:39.687350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64507ac6234948fba1aa5c64216b7a1e', 'request_id': 'req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:39.700624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] The token cache is being invalidate because project 64507ac6234948fba1aa5c64216b7a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:48:39.700624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:39.704787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2cd91523-d241-4fc5-b2aa-b45cf7f74ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:39.704787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1309/2621] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:48:39 2022] DELETE /identity/v3/projects/64507ac6234948fba1aa5c64216b7a1e => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:39.709784 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:39.711551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:39.727553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ae4b6-84b2-4361-8921-bb9ee8a6146c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:39.736029 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:39.794648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:39.795528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:39.795851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:39.796124 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] PATH_INFO: `/v3/users/cd2e522377444450b995b36874f9101b` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:39.797372 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2e522377444450b995b36874f9101b)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:39.799347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:39.954760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2e522377444450b995b36874f9101b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:39.955820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2e522377444450b995b36874f9101b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.002011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2e522377444450b995b36874f9101b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.013505 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2e522377444450b995b36874f9101b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.034433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2e522377444450b995b36874f9101b'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.042850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1313/2622] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:48:39 2022] DELETE /identity/v3/users/cd2e522377444450b995b36874f9101b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:40.154253 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:40.155226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:40.155514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:40.188445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] MFA Rules not processed for user `7e7e03514f454d40aed80b81acab82ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:40.190070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:40.290001 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1310/2623] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:48:40.307765 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:40.309759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:40.380078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:40.381609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:40.411479 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb27a28b-53d7-4c0b-a091-df9c8efcfb98 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:40.418952 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933c7af-f26e-4559-8858-856c60366c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:40.419159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c7af-f26e-4559-8858-856c60366c6f service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:40.419370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c7af-f26e-4559-8858-856c60366c6f service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:40.419610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933c7af-f26e-4559-8858-856c60366c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:40.429613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:40.430944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c7af-f26e-4559-8858-856c60366c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:40.432602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a933c7af-f26e-4559-8858-856c60366c6f service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:40.447254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1314/2624] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:40.479466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd58a27-aec3-42ea-ad33-da9c657d2e48 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:40.487339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:40.488794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:40.489084 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:40.489365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] PATH_INFO: `/v3/projects/bfdee22b8b8d405e87f78d7110aa5cd7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:40.492409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfdee22b8b8d405e87f78d7110aa5cd7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:40.495194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:40.686086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfdee22b8b8d405e87f78d7110aa5cd7', 'request_id': 'req-527d2552-9fdf-4034-bb8c-ea36c21af723'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.830117 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfdee22b8b8d405e87f78d7110aa5cd7', 'request_id': 'req-527d2552-9fdf-4034-bb8c-ea36c21af723'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.843549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfdee22b8b8d405e87f78d7110aa5cd7', 'request_id': 'req-527d2552-9fdf-4034-bb8c-ea36c21af723'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.857440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] The token cache is being invalidate because project bfdee22b8b8d405e87f78d7110aa5cd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:48:40.857740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.858484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:48:40.863765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1311/2625] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:48:40 2022] DELETE /identity/v3/projects/bfdee22b8b8d405e87f78d7110aa5cd7 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:48:43.037112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ec728-6b0f-4175-9772-3e1ba73c3357 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:43.038047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ec728-6b0f-4175-9772-3e1ba73c3357 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:43.038047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ec728-6b0f-4175-9772-3e1ba73c3357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:43.056678 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-235ec728-6b0f-4175-9772-3e1ba73c3357 None None] MFA Rules not processed for user `7e7e03514f454d40aed80b81acab82ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:43.057900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-235ec728-6b0f-4175-9772-3e1ba73c3357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:43.069839 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:43.117614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1315/2626] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:43.435316 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:48:43.436929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:43.499203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d2552-9fdf-4034-bb8c-ea36c21af723 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:48:43.503149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:48:43.504066 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:43.504181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:43.504362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:43.506155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:43.516641 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:48:43.518081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:48:43.526383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1312/2627] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:48:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:48:53.200777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=17 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:48:53.285841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:48:53.286068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:48:53.286349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:48:53.305661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] MFA Rules not processed for user `2bb3e3fc9a494cdf8abd264038af044c`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:48:53.307468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:48:53.359377 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1316/2628] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:48:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:48:53.522648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:03.352400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=25 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:10.893318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:10.894740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:10.905540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19abfd5-de07-4fad-869e-d2e9669d748b service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:10.909167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:10.909794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:10.910078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:10.915819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:10.915819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:10.983600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:10.985080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:10.995272 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1313/2629] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:11.347967 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:11.358144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ed72b-0973-46a7-81e7-103484e34cda None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:11.363354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:11.363860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:11.364141 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:11.365040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:11.366896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:11.435419 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:11.437115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:11.447116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1317/2630] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:15.112271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:15.114453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:15.132230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aa220-088f-4e2f-989a-b039078a223d service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:15.133241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:15.134097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:15.134386 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:15.134661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] PATH_INFO: `/v3/users/3d3896bd70dd43fe95f7125f9f4693a0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:15.135929 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3896bd70dd43fe95f7125f9f4693a0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:15.137987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:15.305546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3896bd70dd43fe95f7125f9f4693a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:15.306191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3896bd70dd43fe95f7125f9f4693a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:15.394018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3896bd70dd43fe95f7125f9f4693a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:15.406696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3896bd70dd43fe95f7125f9f4693a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:15.413350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-1c8b2ed6-166b-49cb-b2ab-271d7df2fcc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3896bd70dd43fe95f7125f9f4693a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:15.419867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1314/2631] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:49:15 2022] DELETE /identity/v3/users/3d3896bd70dd43fe95f7125f9f4693a0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:15.774767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:15.786583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccdaf26-779d-4157-8a09-d5175db599cc service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:15.843489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:15.844382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:15.844692 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:15.844973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] PATH_INFO: `/v3/projects/1e2a93c20b2e4ace88184d8b156a0575` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:15.846154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e2a93c20b2e4ace88184d8b156a0575)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:15.848109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:15.962785 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e2a93c20b2e4ace88184d8b156a0575', 'request_id': 'req-1638a249-6add-4234-bf10-b3ee009af5ec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:16.051746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e2a93c20b2e4ace88184d8b156a0575', 'request_id': 'req-1638a249-6add-4234-bf10-b3ee009af5ec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:16.057497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e2a93c20b2e4ace88184d8b156a0575', 'request_id': 'req-1638a249-6add-4234-bf10-b3ee009af5ec'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:16.068990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] The token cache is being invalidate because project 1e2a93c20b2e4ace88184d8b156a0575 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:49:16.069310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:16.069933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1638a249-6add-4234-bf10-b3ee009af5ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:16.071467 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1318/2632] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:49:15 2022] DELETE /identity/v3/projects/1e2a93c20b2e4ace88184d8b156a0575 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:16.076994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.077286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.077590 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.094893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:16.096124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:16.158836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1315/2633] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:16.166578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.166871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.167152 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.185857 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:16.187068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:16.195042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1319/2634] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:16.199146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.208696 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a7213-a63d-43a3-860f-29f5a0034cc5 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.213295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.213878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.214162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.214440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.215160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:49:16.215468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.217614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.219098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:16.230083 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1316/2635] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:49:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:49:16.235065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.244766 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5fa13-0924-40b3-a89b-ac371fe06846 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.249907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.250559 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.250841 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.251114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.251846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.253817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.268796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:16.321387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1320/2636] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/projects => generated 403 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:16.325856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.335905 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9751d98-1023-4f2a-aa32-6b386e6af895 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.392075 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.392730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.393018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.393292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.393977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.395974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.486642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1317/2637] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/users => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:16.490829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.501172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c926b-df4a-4c15-9726-e8d70471a1c0 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.505832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.506478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.506764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.507040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.507792 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.509604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.520951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1321/2638] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:49:16.524794 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.535327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7278a5a7-6ff6-4cad-987e-f547c937776f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.540082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c5c4d-325b-4e95-9798-1a2a0fc2ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.540730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c5c4d-325b-4e95-9798-1a2a0fc2ae20 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.541014 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c5c4d-325b-4e95-9798-1a2a0fc2ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.541289 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c5c4d-325b-4e95-9798-1a2a0fc2ae20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.542035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c5c4d-325b-4e95-9798-1a2a0fc2ae20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.543649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c5c4d-325b-4e95-9798-1a2a0fc2ae20 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.553455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1318/2639] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:49:16.557404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.567185 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf5ea2-0a44-407b-a62c-2a02f1a364fc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.571807 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.572689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.572971 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.573245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] PATH_INFO: `/v3/projects/fe3a3c8b701d4d86bcc0744dec42d34f/users/ef2834ed53324318810478098889a7ce/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.588195 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3a3c8b701d4d86bcc0744dec42d34f, user_id=ef2834ed53324318810478098889a7ce, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.590968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.663728 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1322/2640] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:49:16 2022] PUT /identity/v3/projects/fe3a3c8b701d4d86bcc0744dec42d34f/users/ef2834ed53324318810478098889a7ce/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:49:16.674350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.674668 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.674947 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.700823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] MFA Rules not processed for user `ef2834ed53324318810478098889a7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:16.702073 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:16.769495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1319/2641] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:16.774071 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.833992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d30f4f-57a7-4a51-8cbf-0ccabfae4b99 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:16.838755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:16.839423 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:16.839738 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:16.840044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:16.841272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:16.843198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:16.974404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1323/2642] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/projects => generated 403 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:16.978461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:16.990497 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14894d7e-3336-48fe-bfb6-25811e32aabc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.046625 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.047203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.047510 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.047817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.048525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.050456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:17.057916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:17.144800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1320/2643] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:49:16 2022] POST /identity/v3/users => generated 308 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:17.148567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:17.159916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f229-bae5-4339-80b2-89bfaebbe7ac admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.164522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.165256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.165611 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.165892 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.166614 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.168237 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:17.179354 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1324/2644] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:49:17.183467 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:17.193622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c9b3b-7738-4ea9-ac68-c0ac0d96e3e4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.198162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.199034 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.199318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.199592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] PATH_INFO: `/v3/projects/40a95b522fd445e5a8cfd5d2384d956f/users/af674f3f24d747f3b1c1a019fa3096f9/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.214802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a95b522fd445e5a8cfd5d2384d956f, user_id=af674f3f24d747f3b1c1a019fa3096f9, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.218190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:17.292270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1321/2645] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:49:17 2022] PUT /identity/v3/projects/40a95b522fd445e5a8cfd5d2384d956f/users/af674f3f24d747f3b1c1a019fa3096f9/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:17.296240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:17.309079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c65e6d-068b-4dc9-bf1f-502d3b70d6ac admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.362565 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeac021-3e69-4eeb-93d4-55de2263c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.363231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac021-3e69-4eeb-93d4-55de2263c0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.363520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac021-3e69-4eeb-93d4-55de2263c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.363832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeac021-3e69-4eeb-93d4-55de2263c0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.364553 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeac021-3e69-4eeb-93d4-55de2263c0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.366154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeac021-3e69-4eeb-93d4-55de2263c0b9 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:17.376951 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1325/2646] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:49:17.381413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:17.392617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead55643-f9a3-4059-8dc6-92b9fd19a036 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.397364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.398235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.398520 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.398799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] PATH_INFO: `/v3/projects/40a95b522fd445e5a8cfd5d2384d956f/users/af674f3f24d747f3b1c1a019fa3096f9/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.400801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a95b522fd445e5a8cfd5d2384d956f, user_id=af674f3f24d747f3b1c1a019fa3096f9, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.403578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:17.413817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1322/2647] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:49:17 2022] PUT /identity/v3/projects/40a95b522fd445e5a8cfd5d2384d956f/users/af674f3f24d747f3b1c1a019fa3096f9/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:17.424871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.425165 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.425444 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.452317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] MFA Rules not processed for user `af674f3f24d747f3b1c1a019fa3096f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:17.453528 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:17.471499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:17.522510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1326/2648] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:17 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:17.533134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:17.534818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:17.591213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af22608-74c4-4956-a318-9dcf1b2dd1e2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.595008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf3c45f-f905-4d25-8b5e-429cb283cc9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.595637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c45f-f905-4d25-8b5e-429cb283cc9d service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.595956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c45f-f905-4d25-8b5e-429cb283cc9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.596235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf3c45f-f905-4d25-8b5e-429cb283cc9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.605381 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3c45f-f905-4d25-8b5e-429cb283cc9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.606780 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf3c45f-f905-4d25-8b5e-429cb283cc9d service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:17.616108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1323/2649] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:17.743318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:17.804108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81224f-e90e-49f8-b330-946d58159b01 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:17.808659 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:17.809517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:17.809835 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:17.810169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] PATH_INFO: `/v3/users/ef2834ed53324318810478098889a7ce` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:17.812033 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef2834ed53324318810478098889a7ce)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:17.813944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:18.009142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef2834ed53324318810478098889a7ce'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:18.009864 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef2834ed53324318810478098889a7ce'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:18.073956 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef2834ed53324318810478098889a7ce'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:18.081171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef2834ed53324318810478098889a7ce'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:18.093729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef2834ed53324318810478098889a7ce'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:18.099506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1327/2650] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:49:17 2022] DELETE /identity/v3/users/ef2834ed53324318810478098889a7ce => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:18.104131 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:18.104477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:18.104753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:18.122982 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:18.124204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:18.188209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1324/2651] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:18.198254 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:18.199714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:18.241567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:18.258607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93220ba2-6f46-4edd-863d-2f6d4f844b5f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:18.264906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:18.264906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:18.264906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:18.264906 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:18.273556 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:18.274981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:18.285573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1328/2652] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:18.743270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:18.743270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:18.753039 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04976a1-78fd-47e2-8a78-457a513afe44 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:18.758891 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:18.759809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:18.760092 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:18.760367 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] PATH_INFO: `/v3/projects/fe3a3c8b701d4d86bcc0744dec42d34f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:18.761603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3a3c8b701d4d86bcc0744dec42d34f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:18.764538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:18.926226 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3a3c8b701d4d86bcc0744dec42d34f', 'request_id': 'req-e924a86d-a025-4631-ae50-f0353742baac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:18.928747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:19.005845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3a3c8b701d4d86bcc0744dec42d34f', 'request_id': 'req-e924a86d-a025-4631-ae50-f0353742baac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.012012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3a3c8b701d4d86bcc0744dec42d34f', 'request_id': 'req-e924a86d-a025-4631-ae50-f0353742baac'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.024141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] The token cache is being invalidate because project fe3a3c8b701d4d86bcc0744dec42d34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:49:19.024366 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.024991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.026595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1325/2653] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:49:18 2022] DELETE /identity/v3/projects/fe3a3c8b701d4d86bcc0744dec42d34f => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:19.030533 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:19.040191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d91c1-59dc-43ad-ac56-c0e2c18822ab service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:19.096184 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:19.097044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:19.097335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:19.097615 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] PATH_INFO: `/v3/users/af674f3f24d747f3b1c1a019fa3096f9` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:19.098798 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af674f3f24d747f3b1c1a019fa3096f9)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:19.100748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:19.283235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af674f3f24d747f3b1c1a019fa3096f9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.283235 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af674f3f24d747f3b1c1a019fa3096f9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.354829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af674f3f24d747f3b1c1a019fa3096f9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.362980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af674f3f24d747f3b1c1a019fa3096f9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.375502 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-5ecd7c69-ca9d-497c-b193-79be6aa330d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af674f3f24d747f3b1c1a019fa3096f9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:19.381856 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1329/2654] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:49:19 2022] DELETE /identity/v3/users/af674f3f24d747f3b1c1a019fa3096f9 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:49:19.861334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:19.862793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:19.875687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e924a86d-a025-4631-ae50-f0353742baac admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:19.932257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:19.933115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:19.933392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:19.933661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] PATH_INFO: `/v3/projects/40a95b522fd445e5a8cfd5d2384d956f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:19.934830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a95b522fd445e5a8cfd5d2384d956f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:19.936913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.097480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a95b522fd445e5a8cfd5d2384d956f', 'request_id': 'req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:20.177835 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a95b522fd445e5a8cfd5d2384d956f', 'request_id': 'req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:20.183807 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a95b522fd445e5a8cfd5d2384d956f', 'request_id': 'req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:20.194820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] The token cache is being invalidate because project 40a95b522fd445e5a8cfd5d2384d956f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:49:20.195134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:20.195803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e68b2d56-5120-4f9a-9c13-dfe7075e7fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:49:20.197348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1326/2655] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:49:19 2022] DELETE /identity/v3/projects/40a95b522fd445e5a8cfd5d2384d956f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:20.202751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.203231 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.203338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.221207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:20.222468 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:20.273788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:20.283624 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1330/2656] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:20.291303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.291593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.291899 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.310097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:20.311325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:20.319571 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1327/2657] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:20.323581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.333687 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63385db-0d32-42e3-b399-7b61e9c9835a None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:20.338866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:20.339510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.339842 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.340125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.340847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:49:20.341153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:20.343339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.354634 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1331/2658] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:49:20.358890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.368127 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafc0d97-1d99-4793-81a1-88f3f75465ba None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:20.372803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:20.373965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.374249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.374525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.375200 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:20.377126 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.464796 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1328/2659] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:49:20 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:20.469190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.479337 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4f044-b43f-4926-abbc-d512952082ad admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:20.536775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:20.537466 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.537753 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.538031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.538726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:20.540722 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.663736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1332/2660] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:49:20 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:20.668159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.679079 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d89a23-1f31-4300-a585-97a8918b62a5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:20.683629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:20.684305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.684591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.684872 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.685591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:20.687236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.697522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1329/2661] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:49:20.701580 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.711481 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb032f6a-043b-49f0-b6ac-9957ba1b35ee admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:20.716154 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b40f5-d846-4957-97fc-ba8eceff57cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:20.717256 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b40f5-d846-4957-97fc-ba8eceff57cc admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.717543 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b40f5-d846-4957-97fc-ba8eceff57cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.717820 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b40f5-d846-4957-97fc-ba8eceff57cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.718537 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b40f5-d846-4957-97fc-ba8eceff57cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:20.720171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b40f5-d846-4957-97fc-ba8eceff57cc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.730098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1333/2662] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:49:20.735687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.747089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d425319-a352-4df7-9a27-526c6876ee44 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:20.751913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:20.752834 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.753115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.753432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] PATH_INFO: `/v3/projects/a9277c44f64846cc840d8edd2fcb803c/users/d2230d6a9cda4ee3b668ac4bef1b3f29/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.777602 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9277c44f64846cc840d8edd2fcb803c, user_id=d2230d6a9cda4ee3b668ac4bef1b3f29, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:20.781753 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:20.852608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1330/2663] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:49:20 2022] PUT /identity/v3/projects/a9277c44f64846cc840d8edd2fcb803c/users/d2230d6a9cda4ee3b668ac4bef1b3f29/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:20.863493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:20.863960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:20.864341 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:20.892094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] MFA Rules not processed for user `d2230d6a9cda4ee3b668ac4bef1b3f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:20.893287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:20.955830 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1334/2664] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:20.960274 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:20.978361 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:21.021513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1c0fe-bdfa-4481-8bcc-f42fa7e32ec4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.026104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.026766 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.027090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.027369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.028111 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.030640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.104448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1331/2665] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:49:20 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:49:21.108544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.119190 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01ad1-0a28-46f0-b145-7b1622070809 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.166643 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:21.175293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.175998 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.176281 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.176563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.177250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.179197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.270089 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1335/2666] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:49:21 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:49:21.274057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.284678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91abcc-e8cf-4679-8b05-f08bf4e979cb admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.289427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.290100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.290375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.290649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.291624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.293383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.303836 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1332/2667] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:49:21.307937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.318378 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fab33d4-fa7d-4cd1-9c8a-c9ecf2f0edf6 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.323005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.324837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.324837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.324837 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] PATH_INFO: `/v3/projects/70934a40220149b19398f6f7665146f2/users/7f21b32bb62f435aa40b5adb263f43b7/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.339360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70934a40220149b19398f6f7665146f2, user_id=7f21b32bb62f435aa40b5adb263f43b7, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.342175 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.405566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1336/2668] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:49:21 2022] PUT /identity/v3/projects/70934a40220149b19398f6f7665146f2/users/7f21b32bb62f435aa40b5adb263f43b7/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:21.409411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.419548 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea19c9e-2368-4ea7-ab83-97dd844c3aaf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.475239 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8d589-2dfd-44f4-922f-bc7c4aa5cbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.475917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8d589-2dfd-44f4-922f-bc7c4aa5cbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.476209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8d589-2dfd-44f4-922f-bc7c4aa5cbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.476485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8d589-2dfd-44f4-922f-bc7c4aa5cbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.477197 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8d589-2dfd-44f4-922f-bc7c4aa5cbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.478779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8d589-2dfd-44f4-922f-bc7c4aa5cbf8 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.489332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1333/2669] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:49:21.493397 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.504348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232ec34-08e1-4150-9705-f29274946277 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.509098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.509966 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.510255 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.510530 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] PATH_INFO: `/v3/projects/70934a40220149b19398f6f7665146f2/users/7f21b32bb62f435aa40b5adb263f43b7/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.512551 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70934a40220149b19398f6f7665146f2, user_id=7f21b32bb62f435aa40b5adb263f43b7, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.515432 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.525561 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1337/2670] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:49:21 2022] PUT /identity/v3/projects/70934a40220149b19398f6f7665146f2/users/7f21b32bb62f435aa40b5adb263f43b7/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:49:21.536840 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.537136 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.537421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.564152 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] MFA Rules not processed for user `7f21b32bb62f435aa40b5adb263f43b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:21.571965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:21.631287 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1334/2671] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:49:21.642775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.644499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:21.703331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19ea16d-636a-4381-92f4-126500995608 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.707059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.707730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.708019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.708296 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.718253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.719689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:21.729302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1338/2672] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:21.922110 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:21.926749 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:21.932352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c41b3d-bbf9-4c32-bbda-9f096dbafe03 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:21.936420 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:21.937052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:21.937335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:21.937613 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:21.992249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:21.993713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:22.003936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1335/2673] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:31.725050 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=46,keystone:INSERT=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:32.034926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=27 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:49:51.681360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:51.682876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:51.755989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697084c-df18-4021-b64c-d75cd7540c3e service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:51.761638 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:51.762293 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:51.762577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:51.762854 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:51.764945 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:51.839194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:51.840834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bd812-e746-426e-a1aa-22b5b5f23702 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:51.851570 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1339/2674] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:57.084399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:49:57.085948 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:57.142689 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d34c1-3469-419d-adc0-12f30a431215 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:49:57.146357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:49:57.146997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:57.147282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:57.147567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:57.150090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:57.220012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:49:57.221557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:49:57.236816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1336/2675] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:49:57.657850 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:49:57.658450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:49:57.658450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:49:57.676840 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] MFA Rules not processed for user `7e7e03514f454d40aed80b81acab82ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:49:57.678253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:49:57.736695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1340/2676] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:49:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:05.789516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:05.794130 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:05.809659 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e992871-e5de-4192-b418-f15e24ad72a5 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:05.815860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:05.816957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:05.817244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:05.817519 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] PATH_INFO: `/v3/users/b13ca365980f4250820a733efe5f44e9` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:05.818729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13ca365980f4250820a733efe5f44e9)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:05.821303 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:05.974740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13ca365980f4250820a733efe5f44e9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:05.975403 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13ca365980f4250820a733efe5f44e9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.033225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13ca365980f4250820a733efe5f44e9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.046507 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13ca365980f4250820a733efe5f44e9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.053552 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13ca365980f4250820a733efe5f44e9'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.060294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1337/2677] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:05 2022] DELETE /identity/v3/users/b13ca365980f4250820a733efe5f44e9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:06.315454 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:06.327715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8d1b8-43f5-46e6-a7f3-5cbd2d0ab10d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:06.370144 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:06.386159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:06.387054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:06.387417 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:06.387712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] PATH_INFO: `/v3/projects/3fc5fd4594f2462fad03b033bbd2aa59` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:06.389166 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fc5fd4594f2462fad03b033bbd2aa59)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:06.391228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:06.491867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fc5fd4594f2462fad03b033bbd2aa59', 'request_id': 'req-3c79c48b-de49-45ae-b861-b884f8283db4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.547865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fc5fd4594f2462fad03b033bbd2aa59', 'request_id': 'req-3c79c48b-de49-45ae-b861-b884f8283db4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.553683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fc5fd4594f2462fad03b033bbd2aa59', 'request_id': 'req-3c79c48b-de49-45ae-b861-b884f8283db4'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.565080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] The token cache is being invalidate because project 3fc5fd4594f2462fad03b033bbd2aa59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:06.565367 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.566048 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.567523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1341/2678] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:06 2022] DELETE /identity/v3/projects/3fc5fd4594f2462fad03b033bbd2aa59 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:06.571579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:06.573334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:06.587438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21cf6b2-48d6-421d-a3f7-dd6840d4e076 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:06.606517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:06.643352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:06.644288 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:06.644569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:06.644847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] PATH_INFO: `/v3/users/d5cda8c2b325488b8d9fec9c5a25bfcb` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:06.646002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5cda8c2b325488b8d9fec9c5a25bfcb)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:06.647916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:06.865516 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5cda8c2b325488b8d9fec9c5a25bfcb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.866244 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5cda8c2b325488b8d9fec9c5a25bfcb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.930954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5cda8c2b325488b8d9fec9c5a25bfcb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.944330 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5cda8c2b325488b8d9fec9c5a25bfcb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.951961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5cda8c2b325488b8d9fec9c5a25bfcb'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:06.958139 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1338/2679] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:06 2022] DELETE /identity/v3/users/d5cda8c2b325488b8d9fec9c5a25bfcb => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:07.189685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:07.191451 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:07.207532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79c48b-de49-45ae-b861-b884f8283db4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:07.278746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:07.279715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:07.280007 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:07.280311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] PATH_INFO: `/v3/projects/9d0e68f11d444e34af4438fc65cc9f66` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:07.281549 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d0e68f11d444e34af4438fc65cc9f66)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:07.283499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:07.376742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d0e68f11d444e34af4438fc65cc9f66', 'request_id': 'req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:07.450985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d0e68f11d444e34af4438fc65cc9f66', 'request_id': 'req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:07.457123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d0e68f11d444e34af4438fc65cc9f66', 'request_id': 'req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:07.469099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] The token cache is being invalidate because project 9d0e68f11d444e34af4438fc65cc9f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:07.469390 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:07.470026 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-f0f9f04a-3042-4317-bf3f-54bcf1b8d1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:07.471610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1342/2680] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:07 2022] DELETE /identity/v3/projects/9d0e68f11d444e34af4438fc65cc9f66 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:07.957225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:07.959686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:07.972976 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941a7d1d-5f13-4376-a2bf-531c60e0b840 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:08.032529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:08.033398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:08.033683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:08.033958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] PATH_INFO: `/v3/users/d2230d6a9cda4ee3b668ac4bef1b3f29` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:08.035141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2230d6a9cda4ee3b668ac4bef1b3f29)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:08.037128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:08.218808 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:08.221735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2230d6a9cda4ee3b668ac4bef1b3f29'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:08.222398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2230d6a9cda4ee3b668ac4bef1b3f29'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:08.285335 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2230d6a9cda4ee3b668ac4bef1b3f29'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:08.297812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2230d6a9cda4ee3b668ac4bef1b3f29'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:08.306190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2230d6a9cda4ee3b668ac4bef1b3f29'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:08.312560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1339/2681] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:07 2022] DELETE /identity/v3/users/d2230d6a9cda4ee3b668ac4bef1b3f29 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:08.317167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:08.317461 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:08.317744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:08.324771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:08.336666 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:08.338440 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:08.397657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1343/2682] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:08.407944 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:08.409374 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:08.466222 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf0558-f205-4ac8-a618-031a9cd9f9a3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:08.470030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:08.470673 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:08.470957 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:08.471238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:08.480546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:08.481991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:08.491881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1340/2683] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:08.875606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:08.878038 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:08.891135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3592f-435c-4445-824d-d7964a6e6408 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:08.895985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:08.897274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:08.897274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:08.897477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] PATH_INFO: `/v3/projects/a9277c44f64846cc840d8edd2fcb803c` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:08.898609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9277c44f64846cc840d8edd2fcb803c)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:08.901181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:09.010965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9277c44f64846cc840d8edd2fcb803c', 'request_id': 'req-984a43c6-ef70-4574-8339-b2c3a5d55caf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.084128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9277c44f64846cc840d8edd2fcb803c', 'request_id': 'req-984a43c6-ef70-4574-8339-b2c3a5d55caf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.089882 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9277c44f64846cc840d8edd2fcb803c', 'request_id': 'req-984a43c6-ef70-4574-8339-b2c3a5d55caf'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.100781 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] The token cache is being invalidate because project a9277c44f64846cc840d8edd2fcb803c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:09.101065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.101690 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.103817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1344/2684] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:08 2022] DELETE /identity/v3/projects/a9277c44f64846cc840d8edd2fcb803c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:09.107860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:09.117394 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa7fb3-70ee-4342-9eca-aa15db378a19 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:09.174879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:09.175829 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:09.176109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:09.176384 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] PATH_INFO: `/v3/users/7f21b32bb62f435aa40b5adb263f43b7` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:09.178072 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f21b32bb62f435aa40b5adb263f43b7)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:09.180050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:09.355990 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f21b32bb62f435aa40b5adb263f43b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.356648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f21b32bb62f435aa40b5adb263f43b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.456302 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f21b32bb62f435aa40b5adb263f43b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.464537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:09.468954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f21b32bb62f435aa40b5adb263f43b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.475690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-52591bc8-c512-4565-9a80-fad3ac356ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f21b32bb62f435aa40b5adb263f43b7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:09.482179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1341/2685] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:09 2022] DELETE /identity/v3/users/7f21b32bb62f435aa40b5adb263f43b7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:09.961631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:09.964132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:09.982167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a43c6-ef70-4574-8339-b2c3a5d55caf admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.053523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.054416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.054700 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.054976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] PATH_INFO: `/v3/projects/70934a40220149b19398f6f7665146f2` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.056310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] RBAC: Authorizing `identity:delete_project(project_id=70934a40220149b19398f6f7665146f2)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:10.060244 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:10.156203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:10.171090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70934a40220149b19398f6f7665146f2', 'request_id': 'req-1f52fa4c-c557-4053-a009-4a5014c2383a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:10.285198 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70934a40220149b19398f6f7665146f2', 'request_id': 'req-1f52fa4c-c557-4053-a009-4a5014c2383a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:10.293720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70934a40220149b19398f6f7665146f2', 'request_id': 'req-1f52fa4c-c557-4053-a009-4a5014c2383a'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:10.305601 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] The token cache is being invalidate because project 70934a40220149b19398f6f7665146f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:10.306079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:10.306804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-1f52fa4c-c557-4053-a009-4a5014c2383a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:10.309241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1345/2686] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:09 2022] DELETE /identity/v3/projects/70934a40220149b19398f6f7665146f2 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:10.314158 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.314643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.314731 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.335624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:10.336861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:10.404283 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1342/2687] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:10.411911 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.412358 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.412823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.440650 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:10.442008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:10.451299 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1346/2688] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:50:10.455718 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:10.466868 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ea865-de46-41f9-a77d-d8dded665cbe None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.474650 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.475369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.475710 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.475993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.476733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:50:10.477044 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:10.479580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:10.494521 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1343/2689] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:50:10.499769 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:10.516715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ef453-0c3c-418e-a4ed-1ef8f864a8dd None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.523263 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.525964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.526250 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.527216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.527937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:10.530069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:10.669955 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1347/2690] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:50:10 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:50:10.674270 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:10.684916 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84769-ad3d-4a5c-8df1-71b26fb52700 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.740781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.741551 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.741819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.742095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.742785 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:10.745248 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:10.871405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1344/2691] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:50:10 2022] POST /identity/v3/users => generated 284 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:10.875404 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:10.892637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3a861-e467-4461-b7be-5ea3749aff41 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.900612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.901696 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.902206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.902736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.904151 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:10.907137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:10.926267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1348/2692] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:50:10.930184 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:10.943300 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec041f-f66a-45ec-8f1a-45a1ac2cd338 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.948697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f88e903-0ac5-44b3-96a4-0d16d84cdaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.949524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88e903-0ac5-44b3-96a4-0d16d84cdaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.949809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88e903-0ac5-44b3-96a4-0d16d84cdaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.950086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f88e903-0ac5-44b3-96a4-0d16d84cdaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.951104 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88e903-0ac5-44b3-96a4-0d16d84cdaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:10.952486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f88e903-0ac5-44b3-96a4-0d16d84cdaaa admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:10.962736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1345/2693] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:10.967269 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:10.977941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351a8b26-10a4-43b2-a167-bbeaa3d4a66a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:10.982623 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:10.983496 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:10.983806 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:10.984081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] PATH_INFO: `/v3/projects/c2eb6b1c62b94b41ab0d09f59c68fb7f/users/4720b5ff1a3c426e8037661475f367a0/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:10.998827 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2eb6b1c62b94b41ab0d09f59c68fb7f, user_id=4720b5ff1a3c426e8037661475f367a0, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.002108 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.085436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1349/2694] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:50:10 2022] PUT /identity/v3/projects/c2eb6b1c62b94b41ab0d09f59c68fb7f/users/4720b5ff1a3c426e8037661475f367a0/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:11.096845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.097164 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.097449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.125513 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] MFA Rules not processed for user `4720b5ff1a3c426e8037661475f367a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:11.126637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:11.167765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:11.193406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1346/2695] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:11.197975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:11.268889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611c09-351c-420a-b37f-583a4071d0f8 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:11.273844 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:11.274575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.275416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.275731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.276455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.278519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.423897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1350/2696] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:50:11 2022] POST /identity/v3/projects => generated 369 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:11.427902 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:11.439269 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc9a6f7-07d5-46f3-b46e-4a77428eb7b1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:11.499258 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:11.499959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.500241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.500524 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.501210 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.503165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.620708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1347/2697] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:50:11 2022] POST /identity/v3/users => generated 291 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:11.621965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:11.638368 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491197b-0884-4cf5-bf5f-f052e64bff3f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:11.643117 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:11.643817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.644104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.644382 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.645098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.646718 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.659402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1351/2698] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:11.665196 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:11.675569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff3a54-f0ec-4010-944e-61e5805bd701 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:11.680404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:11.681801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.682080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.682353 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] PATH_INFO: `/v3/projects/8e117ac2eb9e44ea96245e9e309c3d18/users/80ca8f1f6ffd44b197ff70cac1f54387/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.698725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e117ac2eb9e44ea96245e9e309c3d18, user_id=80ca8f1f6ffd44b197ff70cac1f54387, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.706537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.840639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1348/2699] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:50:11 2022] PUT /identity/v3/projects/8e117ac2eb9e44ea96245e9e309c3d18/users/80ca8f1f6ffd44b197ff70cac1f54387/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:11.844711 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:11.855172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ff2ce4-2847-4c33-b2a8-390f1030b739 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:11.880409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:11.915730 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afabee-be9c-4035-b060-e75aa5606288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:11.916426 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afabee-be9c-4035-b060-e75aa5606288 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.916797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afabee-be9c-4035-b060-e75aa5606288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.917019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afabee-be9c-4035-b060-e75aa5606288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.917744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afabee-be9c-4035-b060-e75aa5606288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.919351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afabee-be9c-4035-b060-e75aa5606288 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.929267 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1352/2700] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:11.933378 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:11.943608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc105c-a967-4cc3-84a2-4226336781bf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:11.948964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:11.953119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.953119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.953119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] PATH_INFO: `/v3/projects/8e117ac2eb9e44ea96245e9e309c3d18/users/80ca8f1f6ffd44b197ff70cac1f54387/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:11.953119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e117ac2eb9e44ea96245e9e309c3d18, user_id=80ca8f1f6ffd44b197ff70cac1f54387, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:11.955790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:11.968082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1349/2701] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:50:11 2022] PUT /identity/v3/projects/8e117ac2eb9e44ea96245e9e309c3d18/users/80ca8f1f6ffd44b197ff70cac1f54387/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:11.980954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:11.980954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:11.980954 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:12.008067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] MFA Rules not processed for user `80ca8f1f6ffd44b197ff70cac1f54387`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:12.009821 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:12.084648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1353/2702] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:11 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:12.096166 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:12.098008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:12.155284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bdb5-7125-4ab5-95da-1da16b689eae admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:12.159040 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:12.159704 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:12.160003 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:12.160281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:12.169873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:12.171310 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:12.182343 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1350/2703] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:12 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:12.277317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:12.278746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:12.291331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6d2bb9-e9ab-455a-ad5f-01a08abd82cb None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:12.295478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b05ec08-062c-4c82-b146-04cbfa75198d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:12.296392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05ec08-062c-4c82-b146-04cbfa75198d service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:12.296677 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05ec08-062c-4c82-b146-04cbfa75198d service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:12.300761 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b05ec08-062c-4c82-b146-04cbfa75198d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:12.356447 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05ec08-062c-4c82-b146-04cbfa75198d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:12.357897 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b05ec08-062c-4c82-b146-04cbfa75198d service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:12.366814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1354/2704] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:17.705537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:17.739031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:17.768448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a16670-68c5-40c9-98aa-cf83cfbad756 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:17.772995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:17.773851 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:17.774160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:17.774435 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] PATH_INFO: `/v3/users/4720b5ff1a3c426e8037661475f367a0` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:17.775606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4720b5ff1a3c426e8037661475f367a0)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:17.777534 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:17.980058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4720b5ff1a3c426e8037661475f367a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:17.980727 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4720b5ff1a3c426e8037661475f367a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.039505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4720b5ff1a3c426e8037661475f367a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.052928 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4720b5ff1a3c426e8037661475f367a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.060354 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4720b5ff1a3c426e8037661475f367a0'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.066589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1351/2705] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:17 2022] DELETE /identity/v3/users/4720b5ff1a3c426e8037661475f367a0 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:18.071135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:18.072181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:18.072344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:18.090482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:18.091733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:18.134834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:18.141327 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:18.142983 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:18.154168 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1355/2706] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:18.163486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:18.165793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:18.218307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b15aa-24c5-4fd9-a188-56e88cd5c4e7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:18.224243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:18.224676 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa67a28-ce39-44a2-97db-c6cf4dcd96f9 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:18.224911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:18.225199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:18.225478 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:18.229559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:18.231881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:18.232200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:18.232486 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:18.232762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:18.242890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:18.243135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:18.244575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:18.244708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:18.255509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1352/2707] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:50:18.255804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1356/2708] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:50:18.624593 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:18.627341 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:18.642337 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769cd2e-dac7-4897-89d9-85247af4d1f8 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:18.646962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:18.648993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:18.649502 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:18.649782 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] PATH_INFO: `/v3/projects/c2eb6b1c62b94b41ab0d09f59c68fb7f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:18.651036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2eb6b1c62b94b41ab0d09f59c68fb7f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:18.652980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:18.752633 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:18.761099 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2eb6b1c62b94b41ab0d09f59c68fb7f', 'request_id': 'req-369b67c2-325f-4da1-b40b-a6b11f584200'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.839305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2eb6b1c62b94b41ab0d09f59c68fb7f', 'request_id': 'req-369b67c2-325f-4da1-b40b-a6b11f584200'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.845338 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2eb6b1c62b94b41ab0d09f59c68fb7f', 'request_id': 'req-369b67c2-325f-4da1-b40b-a6b11f584200'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.856772 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] The token cache is being invalidate because project c2eb6b1c62b94b41ab0d09f59c68fb7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:18.858846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.858846 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:18.859236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1353/2709] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:18 2022] DELETE /identity/v3/projects/c2eb6b1c62b94b41ab0d09f59c68fb7f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:50:18.863197 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:18.873172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74507c-00b9-4640-a88b-2df05bce80a9 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:18.934497 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:18.935375 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:18.935680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:18.935968 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] PATH_INFO: `/v3/users/80ca8f1f6ffd44b197ff70cac1f54387` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:18.937170 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80ca8f1f6ffd44b197ff70cac1f54387)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:18.939289 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:19.133223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80ca8f1f6ffd44b197ff70cac1f54387'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.134498 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80ca8f1f6ffd44b197ff70cac1f54387'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.199149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80ca8f1f6ffd44b197ff70cac1f54387'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.207418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80ca8f1f6ffd44b197ff70cac1f54387'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.220065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-db650c60-2c01-4e01-89fb-29a5eab59488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80ca8f1f6ffd44b197ff70cac1f54387'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.226278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1357/2710] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:18 2022] DELETE /identity/v3/users/80ca8f1f6ffd44b197ff70cac1f54387 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:19.630165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:19.630165 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:19.639275 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369b67c2-325f-4da1-b40b-a6b11f584200 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:19.697224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:19.703025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:19.703025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:19.703025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] PATH_INFO: `/v3/projects/8e117ac2eb9e44ea96245e9e309c3d18` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:19.703025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e117ac2eb9e44ea96245e9e309c3d18)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:19.703025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:19.800292 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e117ac2eb9e44ea96245e9e309c3d18', 'request_id': 'req-d4277b8f-3a36-441e-b37e-9146e907cec5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.850635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e117ac2eb9e44ea96245e9e309c3d18', 'request_id': 'req-d4277b8f-3a36-441e-b37e-9146e907cec5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.856793 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e117ac2eb9e44ea96245e9e309c3d18', 'request_id': 'req-d4277b8f-3a36-441e-b37e-9146e907cec5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.867685 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] The token cache is being invalidate because project 8e117ac2eb9e44ea96245e9e309c3d18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:19.867984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.869180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d4277b8f-3a36-441e-b37e-9146e907cec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:19.870734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1354/2711] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:19 2022] DELETE /identity/v3/projects/8e117ac2eb9e44ea96245e9e309c3d18 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:19.877058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:19.877344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:19.877618 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:19.896348 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:19.897593 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:19.903813 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:19.956828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1358/2712] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:19.965917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:19.967019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:19.967019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:19.984819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:19.986031 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:19.994860 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1355/2713] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:19.999206 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.009586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7ae26-fb52-4874-8873-9a7cdc79c810 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.014453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac06662-f465-4e28-b579-57811712974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.015112 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06662-f465-4e28-b579-57811712974a admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.015395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06662-f465-4e28-b579-57811712974a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.015725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac06662-f465-4e28-b579-57811712974a admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.016424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac06662-f465-4e28-b579-57811712974a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:50:20.016736 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac06662-f465-4e28-b579-57811712974a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.018344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac06662-f465-4e28-b579-57811712974a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.030492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1359/2714] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:50:20.034940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.044738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89d354-17a4-4ccc-9ce7-80ed4599763d None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.049438 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.050082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.050365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.050643 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.051342 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.053339 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.132468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1356/2715] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:50:20 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:20.137476 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac06662-f465-4e28-b579-57811712974a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.148135 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac06662-f465-4e28-b579-57811712974a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.217900 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.218575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.218865 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.219149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.219868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.221815 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.336647 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1360/2716] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:50:20 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:20.341395 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.351448 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0517fbf4-292f-42f9-8a9c-cd65c6eca4b4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.356488 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.357134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.357432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.357687 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.358399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.360030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.371159 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1357/2717] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:20.375276 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.385548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250981c-c87e-45ae-8061-15cb1c9eec88 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.390473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a15a3-cb1e-42ad-91eb-a08e956093c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.391132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a15a3-cb1e-42ad-91eb-a08e956093c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.391420 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a15a3-cb1e-42ad-91eb-a08e956093c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.391729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a15a3-cb1e-42ad-91eb-a08e956093c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.392485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a15a3-cb1e-42ad-91eb-a08e956093c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.394082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a15a3-cb1e-42ad-91eb-a08e956093c1 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.404138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1361/2718] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:20.408358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.418800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd44f7c-7634-4550-bea5-3ed17e2fb0e5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.423242 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.424141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.424421 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.424700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] PATH_INFO: `/v3/projects/1c08c81a773747d291838cd8b6d24dde/users/7a98dd0ed36c4a34a35200a7b1b9ca35/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.438461 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:20.441570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c08c81a773747d291838cd8b6d24dde, user_id=7a98dd0ed36c4a34a35200a7b1b9ca35, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.444285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.571203 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1358/2719] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:50:20 2022] PUT /identity/v3/projects/1c08c81a773747d291838cd8b6d24dde/users/7a98dd0ed36c4a34a35200a7b1b9ca35/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:20.581779 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.582180 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.582470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.618422 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] MFA Rules not processed for user `7a98dd0ed36c4a34a35200a7b1b9ca35`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:20.627997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:20.686829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1362/2720] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:20.696100 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.698754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:20.755688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3826880-1927-4d90-944a-fe378cc9e6fd admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.759325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2281f77-609d-44a8-8a01-b16162fadfc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.759994 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281f77-609d-44a8-8a01-b16162fadfc1 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.760285 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281f77-609d-44a8-8a01-b16162fadfc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.760558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2281f77-609d-44a8-8a01-b16162fadfc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.769095 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2281f77-609d-44a8-8a01-b16162fadfc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.770523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2281f77-609d-44a8-8a01-b16162fadfc1 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:20.779828 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1359/2721] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:20.799982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:20.805374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:20.862176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb4d95-24c0-446c-9118-971235b4152c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:20.867155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:20.868344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:20.869499 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:20.869791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] PATH_INFO: `/v3/users/7a98dd0ed36c4a34a35200a7b1b9ca35` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:20.871064 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a98dd0ed36c4a34a35200a7b1b9ca35)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:20.873106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:21.036086 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a98dd0ed36c4a34a35200a7b1b9ca35'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.036756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a98dd0ed36c4a34a35200a7b1b9ca35'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.102239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a98dd0ed36c4a34a35200a7b1b9ca35'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.110599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a98dd0ed36c4a34a35200a7b1b9ca35'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.123287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a98dd0ed36c4a34a35200a7b1b9ca35'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.129311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1363/2722] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:50:20 2022] DELETE /identity/v3/users/7a98dd0ed36c4a34a35200a7b1b9ca35 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:21.134046 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:21.134517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:21.134517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:21.152695 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:21.153894 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:21.212724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1360/2723] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:50:21.260352 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:21.261833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:21.320485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec61909-9d31-45d0-8147-eeb6e036e911 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:21.324051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ecbb2-0144-487e-818c-c3544918c46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:21.324699 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecbb2-0144-487e-818c-c3544918c46b service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:21.324986 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecbb2-0144-487e-818c-c3544918c46b service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:21.325265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ecbb2-0144-487e-818c-c3544918c46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:21.334361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ecbb2-0144-487e-818c-c3544918c46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:21.335828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ecbb2-0144-487e-818c-c3544918c46b service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:21.345349 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1364/2724] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:21.709406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:21.710845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:21.723064 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337a06-e7f2-4292-811a-a2ca3d46de0b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:21.727823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:21.728706 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:21.728999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:21.729284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] PATH_INFO: `/v3/projects/1c08c81a773747d291838cd8b6d24dde` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:21.730517 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c08c81a773747d291838cd8b6d24dde)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:21.732476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:21.811021 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c08c81a773747d291838cd8b6d24dde', 'request_id': 'req-b7cb4241-9c05-4c30-9323-2304024af282'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.856669 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c08c81a773747d291838cd8b6d24dde', 'request_id': 'req-b7cb4241-9c05-4c30-9323-2304024af282'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.863311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c08c81a773747d291838cd8b6d24dde', 'request_id': 'req-b7cb4241-9c05-4c30-9323-2304024af282'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.875107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] The token cache is being invalidate because project 1c08c81a773747d291838cd8b6d24dde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:50:21.875398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.876070 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b7cb4241-9c05-4c30-9323-2304024af282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:50:21.877693 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1361/2725] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:50:21 2022] DELETE /identity/v3/projects/1c08c81a773747d291838cd8b6d24dde => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:21.882571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:21.882867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:21.883153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:21.901557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:21.902771 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:21.926818 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:21.965332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1365/2726] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:21.973119 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:21.973408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:21.973683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:21.992725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:21.993978 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:22.002959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1362/2727] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:22.007300 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.017789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36a743-61ce-4422-8539-5965bee1e28f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.022799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.023478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.023801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.024082 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.024797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:50:22.025099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.026702 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.038627 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1366/2728] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:50:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:50:22.043621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.053043 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e53f2a-b467-4e60-bd0c-2e4c8d4dce07 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.058406 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.059054 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.059340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.059615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.060375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.062307 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.070570 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:22.178105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1363/2729] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:50:22 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:22.182888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.193105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bce4c6-1424-4e0a-81ac-27793ace0dc3 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.253548 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.254222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.254512 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.254816 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.255521 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.257506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.360149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1367/2730] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:50:22 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:50:22.364387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.374986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624aa76-2378-43af-a7f1-487b5ae2fdd3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.379574 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.380265 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.380578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.380817 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.381561 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.383135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.393512 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1364/2731] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:22.397672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.408474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e331836-2a0b-4844-8a3d-3ebd269bd3ac admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.413143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32df5cc-c174-4892-ac62-d0173268e4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.414531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df5cc-c174-4892-ac62-d0173268e4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.414531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df5cc-c174-4892-ac62-d0173268e4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.414531 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32df5cc-c174-4892-ac62-d0173268e4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.415076 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32df5cc-c174-4892-ac62-d0173268e4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.417386 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32df5cc-c174-4892-ac62-d0173268e4e8 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.429909 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1368/2732] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:50:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:50:22.434810 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.445107 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becdcad0-c8b5-4443-9a16-667c053807f5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.450898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.451904 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.452193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.452476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] PATH_INFO: `/v3/projects/411e8573a7b94719bc65378204cbc53d/users/1b4faf2daf884d7fb15cda3bcd17c074/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.467345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=411e8573a7b94719bc65378204cbc53d, user_id=1b4faf2daf884d7fb15cda3bcd17c074, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.470071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.548792 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1365/2733] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:50:22 2022] PUT /identity/v3/projects/411e8573a7b94719bc65378204cbc53d/users/1b4faf2daf884d7fb15cda3bcd17c074/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:50:22.554044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b52809-33a2-48ce-bb3a-432c519d5ccc None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.556425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b52809-33a2-48ce-bb3a-432c519d5ccc None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.556425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b52809-33a2-48ce-bb3a-432c519d5ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.573094 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-69b52809-33a2-48ce-bb3a-432c519d5ccc None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:22.574286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-69b52809-33a2-48ce-bb3a-432c519d5ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:22.635162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1369/2734] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:22.645273 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:22.647002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:22.705207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a84dc-ce45-45cd-bfb8-fbcb246988b3 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:22.708816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:22.709456 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:22.709742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:22.710025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:22.720052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:22.721498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:22.731370 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1366/2735] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:23.892438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9c42e-4172-4e62-a108-cd2716d4eb02 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:23.892942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9c42e-4172-4e62-a108-cd2716d4eb02 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:23.893054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9c42e-4172-4e62-a108-cd2716d4eb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:23.927444 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-55e9c42e-4172-4e62-a108-cd2716d4eb02 None None] MFA Rules not processed for user `1b4faf2daf884d7fb15cda3bcd17c074`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:23.928720 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-55e9c42e-4172-4e62-a108-cd2716d4eb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:23.952742 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:24.007983 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1370/2736] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:24.015855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:50:24.017477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:24.079802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f95e4b-1a6a-4d46-9be1-0b14768085a6 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:50:24.083447 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d98d3-6cdb-4e89-b74b-4bb0b83437d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:50:24.084161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d98d3-6cdb-4e89-b74b-4bb0b83437d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:24.084449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d98d3-6cdb-4e89-b74b-4bb0b83437d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:24.084726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d98d3-6cdb-4e89-b74b-4bb0b83437d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:24.094418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d98d3-6cdb-4e89-b74b-4bb0b83437d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:50:24.095934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d98d3-6cdb-4e89-b74b-4bb0b83437d3 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:50:24.106365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1367/2737] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:50:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:50:25.767450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88ae9-f4b4-4eb1-ab18-5a5007e8c0cd None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:50:25.768054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88ae9-f4b4-4eb1-ab18-5a5007e8c0cd None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:50:25.768054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df88ae9-f4b4-4eb1-ab18-5a5007e8c0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:50:25.786156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6df88ae9-f4b4-4eb1-ab18-5a5007e8c0cd None None] MFA Rules not processed for user `1b4faf2daf884d7fb15cda3bcd17c074`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:50:25.787365 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6df88ae9-f4b4-4eb1-ab18-5a5007e8c0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:50:25.794402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1371/2738] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:50:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:50:34.104658 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=71,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:50:35.780725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=20 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:51:03.531027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02604c7-f1a3-4c89-8e21-58b96da2636a None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:51:03.531823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02604c7-f1a3-4c89-8e21-58b96da2636a None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:51:03.531823 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02604c7-f1a3-4c89-8e21-58b96da2636a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:51:03.548923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d02604c7-f1a3-4c89-8e21-58b96da2636a None None] MFA Rules not processed for user `7e7e03514f454d40aed80b81acab82ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:51:03.550115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d02604c7-f1a3-4c89-8e21-58b96da2636a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:51:03.604369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1368/2739] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:51:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:51:13.597784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=25 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:51:22.030642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:51:22.031311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:51:22.031311 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:51:22.049222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] MFA Rules not processed for user `2bb3e3fc9a494cdf8abd264038af044c`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:51:22.050550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:51:22.115805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1372/2740] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:51:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:51:32.104767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=25 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:51:36.058609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:51:36.059155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:51:36.059271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:51:36.076870 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] MFA Rules not processed for user `1b4faf2daf884d7fb15cda3bcd17c074`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:51:36.078055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:51:36.085382 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1369/2741] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:51:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:51:46.071409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:09.312470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:09.313871 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:09.326183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9469e046-996a-4321-b8f4-369a2cd1e52e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:09.329796 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:09.330458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:09.330746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:09.331022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:09.332867 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:09.403721 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:09.405157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb5133-0940-4fc5-8c77-af842ff7d8fe service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:09.415964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1373/2742] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:52:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:52:12.182977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:12.192811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78228c1d-eef4-435b-b06e-82d39cf78661 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:12.197121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:12.197759 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:12.198045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:12.198322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:12.200108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:12.215416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:12.216893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:12.228567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1370/2743] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:52:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:52:19.411159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=34 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:22.223181 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=6 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:26.134446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:26.137662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:26.137662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:26.153057 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] MFA Rules not processed for user `7e7e03514f454d40aed80b81acab82ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:52:26.154230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:26.160978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1374/2744] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:52:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:52:36.147744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:42.072924 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:42.080128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f440829f-acf5-40df-b187-59f16b4a29b4 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:42.085849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:42.087007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:42.087007 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:42.087286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] PATH_INFO: `/v3/users/1b4faf2daf884d7fb15cda3bcd17c074` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:42.088580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b4faf2daf884d7fb15cda3bcd17c074)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:42.091225 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:42.274002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b4faf2daf884d7fb15cda3bcd17c074'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.274661 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b4faf2daf884d7fb15cda3bcd17c074'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.350557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b4faf2daf884d7fb15cda3bcd17c074'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.365346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b4faf2daf884d7fb15cda3bcd17c074'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.372559 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-cfc802e4-6094-4c45-bbad-565b5e2ced68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b4faf2daf884d7fb15cda3bcd17c074'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.378671 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1371/2745] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:52:42 2022] DELETE /identity/v3/users/1b4faf2daf884d7fb15cda3bcd17c074 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:52:42.644093 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:42.655959 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5aa087-0b7d-416d-9197-4f0dc07bc836 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:42.718201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:42.719090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:42.719381 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:42.719679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] PATH_INFO: `/v3/projects/411e8573a7b94719bc65378204cbc53d` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:42.720862 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=411e8573a7b94719bc65378204cbc53d)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:42.722768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:42.845576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411e8573a7b94719bc65378204cbc53d', 'request_id': 'req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.927889 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411e8573a7b94719bc65378204cbc53d', 'request_id': 'req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.934795 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411e8573a7b94719bc65378204cbc53d', 'request_id': 'req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.946838 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] The token cache is being invalidate because project 411e8573a7b94719bc65378204cbc53d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:52:42.947125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.947801 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-957b0f3d-0a21-46a1-b76b-e7d2c86d1a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:42.950011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1375/2746] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:52:42 2022] DELETE /identity/v3/projects/411e8573a7b94719bc65378204cbc53d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:52:42.955077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:42.955486 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:42.955841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:42.976895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:52:42.978199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:43.038028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1372/2747] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:52:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:52:43.045975 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.046302 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.046586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.065075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:52:43.066359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:43.075262 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1376/2748] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:52:43.079549 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.089259 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937bb11-e971-45b9-b023-2b4ee60c0a65 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.094506 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.095156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.095437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.095742 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.096522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:52:43.097033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.100052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.116845 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1373/2749] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:52:43.121131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.130646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687abe83-9cd6-4ee9-aa50-33d4f8ed3f9b None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.136326 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.137575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.137924 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.138294 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.139055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.141061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.266207 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1377/2750] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:52:43 2022] POST /identity/v3/projects => generated 395 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:52:43.268154 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.281611 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a0472b-7550-4eaa-9559-03c016368425 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.343465 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.344179 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.344469 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.344752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.345455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.347376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.354255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:43.446271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1374/2751] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:52:43 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:52:43.448582 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.460159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3611a4-1647-4de0-b124-450049e9e2e7 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.465200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.465870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.466163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.466439 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.467163 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.468828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.481772 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1378/2752] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:52:43.485603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.496578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd696db4-68b9-4983-86e9-b3c511702dc5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.502033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220da1a9-5069-4e67-9c4e-8dac903de3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.502784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220da1a9-5069-4e67-9c4e-8dac903de3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.503172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220da1a9-5069-4e67-9c4e-8dac903de3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.503526 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220da1a9-5069-4e67-9c4e-8dac903de3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.504369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220da1a9-5069-4e67-9c4e-8dac903de3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.506235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220da1a9-5069-4e67-9c4e-8dac903de3e9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.518942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1375/2753] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:52:43.523054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.535602 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890384f6-5cd9-4a6b-91ae-7effba9cbc84 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.539655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.540577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.540886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.541503 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] PATH_INFO: `/v3/projects/c61383c8e48945febac054ac36e97879/users/1c0f5c2c2aa14c07961d02779eddf604/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.558114 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61383c8e48945febac054ac36e97879, user_id=1c0f5c2c2aa14c07961d02779eddf604, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.560914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.625717 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1379/2754] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:52:43 2022] PUT /identity/v3/projects/c61383c8e48945febac054ac36e97879/users/1c0f5c2c2aa14c07961d02779eddf604/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:52:43.637322 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.637615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.637900 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.669744 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] MFA Rules not processed for user `1c0f5c2c2aa14c07961d02779eddf604`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:52:43.671149 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:43.740954 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1376/2755] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:52:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:52:43.755922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:43.757976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:43.802668 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:43.839215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b33e45-6b96-4dec-9847-484b9aa31890 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:43.839215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334628b0-94b6-447f-8370-ccef00bde257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:43.839215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334628b0-94b6-447f-8370-ccef00bde257 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:43.839215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334628b0-94b6-447f-8370-ccef00bde257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:43.839829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334628b0-94b6-447f-8370-ccef00bde257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:43.847338 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334628b0-94b6-447f-8370-ccef00bde257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:43.848829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334628b0-94b6-447f-8370-ccef00bde257 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:43.858683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1380/2756] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:52:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:52:51.748365 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:51.809741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3eb7d7-6420-4628-804c-722e12376375 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:51.814192 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:51.815042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:51.815324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:51.815596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] PATH_INFO: `/v3/users/1c0f5c2c2aa14c07961d02779eddf604` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:51.816790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0f5c2c2aa14c07961d02779eddf604)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:51.818684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:51.969775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0f5c2c2aa14c07961d02779eddf604'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:51.970414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0f5c2c2aa14c07961d02779eddf604'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.047325 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0f5c2c2aa14c07961d02779eddf604'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.058229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:52:52.059380 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0f5c2c2aa14c07961d02779eddf604'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.066188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0f5c2c2aa14c07961d02779eddf604'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.072035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1377/2757] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:52:51 2022] DELETE /identity/v3/users/1c0f5c2c2aa14c07961d02779eddf604 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:52:52.077129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:52.078759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:52.078759 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:52.095171 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:52:52.099279 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:52.154122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1381/2758] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:52:52.163820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:52.165204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:52.221657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5798a-ae04-447f-8b1b-ea4fc656f272 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:52.225278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:52.225911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:52.226191 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:52.226466 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:52.235694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:52.237120 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:52.246430 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1378/2759] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:52:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:52:52.660017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:52:52.661522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:52:52.675181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af4f1b-81d1-4be5-8ae7-ad1f51ab409c None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:52:52.679925 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:52:52.681333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:52:52.681758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:52:52.682054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] PATH_INFO: `/v3/projects/c61383c8e48945febac054ac36e97879` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:52:52.683268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61383c8e48945febac054ac36e97879)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:52:52.686140 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:52:52.785188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61383c8e48945febac054ac36e97879', 'request_id': 'req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.858657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61383c8e48945febac054ac36e97879', 'request_id': 'req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.864535 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61383c8e48945febac054ac36e97879', 'request_id': 'req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.875383 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] The token cache is being invalidate because project c61383c8e48945febac054ac36e97879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:52:52.875653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.876304 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:52:52.877829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1382/2760] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:52:52 2022] DELETE /identity/v3/projects/c61383c8e48945febac054ac36e97879 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:02.242252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=32 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:02.872448 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:20.868359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:20.869758 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:20.880861 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6a024-110c-4fc3-88ac-e543394d4a5e service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:20.929726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:20.930360 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:20.930640 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:20.930911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:20.932694 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:20.941740 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:20.943138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71615e2-b588-4199-9962-d79a95c7b79e service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:20.950941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1379/2761] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:30.947364 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=26 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:31.907552 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:31.908969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:31.964176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4371a795-d7c3-47e4-8f85-c9a1bdb9754e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:31.967741 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:31.968374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:31.968660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:31.968937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:31.970672 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:32.027351 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:32.028833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:32.038126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1383/2762] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:34.239295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:34.239991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:34.239991 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:34.258396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] MFA Rules not processed for user `2bb3e3fc9a494cdf8abd264038af044c`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:34.259604 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:34.267329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1380/2763] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:53:34.584817 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:34.586245 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:34.597674 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe842c-63a1-4a32-8b10-e6af23f9abe8 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:34.601191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:34.601824 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:34.602107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:34.602385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:34.604123 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:34.613665 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:34.615088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:34.622905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1384/2764] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:53:35.294825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:35.296295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:35.370036 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07588cf8-ec7d-4bf8-9399-420df0e8e7df None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:35.375033 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:35.375691 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:35.375988 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:35.376267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:35.378057 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:35.389500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:35.391565 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:35.401498 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1381/2765] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:36.736137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:36.737515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:36.837470 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340cb25-2b9c-4883-90f8-c913bb31a996 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:36.843405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:36.845362 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:36.845639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:36.845938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:36.848035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:36.864922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:36.867176 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:36.889985 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1385/2766] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:41.514579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:41.524019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f308a5-11cd-4737-8330-2c7c1ef6bc2a service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:41.530582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:41.530582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:41.530582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:41.530582 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:41.530993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:41.587920 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:41.589346 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:41.598344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1382/2767] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:53:43.278155 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:43.279915 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:43.291907 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1819a5e5-73bf-414c-81d9-ca4b96d5ec77 service nova] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:43.296042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:43.296804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:43.297179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:43.297557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:43.299424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:43.325077 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:43.363395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:43.364973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:43.384776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1386/2768] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:46.759106 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:46.760841 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:46.775956 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ff41a-6caa-40c5-a838-40a1bd4b49a5 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:46.780962 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:46.782363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:46.782660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:46.782941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:46.784736 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:46.797387 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:46.799455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:46.819115 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1383/2769] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:46.907755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:46.907755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:46.927159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bcb92-0d2c-451c-a35f-6486238a485b service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:46.933161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:46.934040 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:46.934321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:46.934596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] PATH_INFO: `/v3/users/7e7e03514f454d40aed80b81acab82ca` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:46.936228 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e7e03514f454d40aed80b81acab82ca)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:46.938973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:47.129921 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e7e03514f454d40aed80b81acab82ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.130588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e7e03514f454d40aed80b81acab82ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.195465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e7e03514f454d40aed80b81acab82ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.203130 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e7e03514f454d40aed80b81acab82ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.215150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e43ac3c1-5902-4b77-961b-438bf78dbe69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e7e03514f454d40aed80b81acab82ca'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.221733 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1387/2770] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:53:46 2022] DELETE /identity/v3/users/7e7e03514f454d40aed80b81acab82ca => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:47.443249 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:47.455562 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391f044-392e-4384-ac95-9dc5dfa42bb4 service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:47.513965 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:47.517429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:47.517429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:47.517429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] PATH_INFO: `/v3/projects/19e92b8219814afd94d3d49bf9f13f30` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:47.517429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e92b8219814afd94d3d49bf9f13f30)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:47.518455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:47.528922 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=100 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:47.628996 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e92b8219814afd94d3d49bf9f13f30', 'request_id': 'req-b93a9931-1a2e-457c-ab31-58f9e119b480'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.702529 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e92b8219814afd94d3d49bf9f13f30', 'request_id': 'req-b93a9931-1a2e-457c-ab31-58f9e119b480'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.708172 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e92b8219814afd94d3d49bf9f13f30', 'request_id': 'req-b93a9931-1a2e-457c-ab31-58f9e119b480'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.719492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] The token cache is being invalidate because project 19e92b8219814afd94d3d49bf9f13f30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:53:47.719797 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.720414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b93a9931-1a2e-457c-ab31-58f9e119b480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:47.722045 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1384/2771] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:53:47 2022] DELETE /identity/v3/projects/19e92b8219814afd94d3d49bf9f13f30 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:47.727084 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:47.729399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:47.729399 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:47.745606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:47.746809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:47.804770 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1388/2772] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:47.813262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:47.813999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:47.813999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:47.831912 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:47.833098 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:47.841167 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1385/2773] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:47.845683 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:47.855075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e2e5d-8937-4203-8aa6-b371c6dc59c5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:47.860241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:47.860880 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:47.861156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:47.861425 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:47.862128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:53:47.862433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:47.864055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:47.875200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1389/2774] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:53:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 06 20:53:47.879228 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:47.888725 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f437b1-2e05-4fa6-9018-6c1b565e58ce None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:47.893483 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:47.894122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:47.894405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:47.894683 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:47.895363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:47.897309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:48.024624 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1386/2775] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:53:47 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:48.028706 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:48.039230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78e1bdd-a45c-498e-8e55-919b395b4194 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:48.055490 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:48.093478 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:48.094941 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.095044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.095248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.095963 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:48.097965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:48.214876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1390/2776] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:53:48 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:53:48.216626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:48.228071 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175801b-72f5-453f-9e1b-85cc5a953f0d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:48.239223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:48.239223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.239223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.239223 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.240446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:48.240446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:48.249618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1387/2777] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:48.253595 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:48.263851 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e495a96-8cbb-40fd-a3c7-d051f9a9292e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:48.268989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02582ac-cc92-4695-8705-ff7bc774568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:48.269639 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02582ac-cc92-4695-8705-ff7bc774568d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.269919 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02582ac-cc92-4695-8705-ff7bc774568d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.270193 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02582ac-cc92-4695-8705-ff7bc774568d admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.270910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02582ac-cc92-4695-8705-ff7bc774568d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:48.272520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02582ac-cc92-4695-8705-ff7bc774568d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:48.285885 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1391/2778] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:48.289842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:48.299690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7a8bb4-6375-4ea5-b6ff-ff6cb56cb575 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:48.304162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:48.305026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.305309 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.305583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] PATH_INFO: `/v3/projects/0a814a9a1f2548d186ddb8b8ca826698/users/8bb1a86fd0984338bd6e6a829266eb91/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.319752 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a814a9a1f2548d186ddb8b8ca826698, user_id=8bb1a86fd0984338bd6e6a829266eb91, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:48.322362 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:48.383862 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1388/2779] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:48 2022] PUT /identity/v3/projects/0a814a9a1f2548d186ddb8b8ca826698/users/8bb1a86fd0984338bd6e6a829266eb91/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:48.394563 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.394853 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.395132 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.421203 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] MFA Rules not processed for user `8bb1a86fd0984338bd6e6a829266eb91`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:48.422412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:48.485927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1392/2780] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:48.496747 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:48.498453 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:48.571291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe49cd9-a62b-40da-86c1-0ceaf0908192 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:48.576734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:48.577368 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.577655 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.577934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.586170 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:48.587804 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:48.596410 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1389/2781] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:48.790990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:48.859224 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57d967-9fa6-4721-87d3-aecb2e22729e None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:48.864177 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:48.865073 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:48.865361 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:48.865631 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] PATH_INFO: `/v3/users/8bb1a86fd0984338bd6e6a829266eb91` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:48.866789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bb1a86fd0984338bd6e6a829266eb91)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:48.868755 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:49.010262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:49.021525 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43441ffe-42e0-42b3-b647-cec9fd813f78 service nova] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:49.027150 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:49.027858 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:49.028229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:49.028509 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:49.030278 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:49.041019 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:49.042500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:49.044207 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde6bc6-d3f4-4cdf-8d0c-8992ca3f0e04 service nova] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:49.055534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bb1a86fd0984338bd6e6a829266eb91'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:49.055726 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1390/2782] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:53:49.056588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bb1a86fd0984338bd6e6a829266eb91'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:49.116508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bb1a86fd0984338bd6e6a829266eb91'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:49.125799 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bb1a86fd0984338bd6e6a829266eb91'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:49.138017 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bb1a86fd0984338bd6e6a829266eb91'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:49.144143 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1393/2783] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:53:48 2022] DELETE /identity/v3/users/8bb1a86fd0984338bd6e6a829266eb91 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:49.148606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:49.154140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:49.154140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:49.161597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:49.167485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:49.168750 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:49.228712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1391/2784] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:49.238846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:49.240946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:49.305747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d8ba7-c827-4671-bc95-8d8471721495 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:49.309515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b486c4bd-bee3-44cd-919e-24fccadcfc46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:49.310162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486c4bd-bee3-44cd-919e-24fccadcfc46 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:49.310452 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486c4bd-bee3-44cd-919e-24fccadcfc46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:49.310726 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b486c4bd-bee3-44cd-919e-24fccadcfc46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:49.320507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486c4bd-bee3-44cd-919e-24fccadcfc46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:49.321994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b486c4bd-bee3-44cd-919e-24fccadcfc46 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:49.332888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1394/2785] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:49.853484 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:49.855019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:49.872941 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6761e939-b237-4447-944f-ed158dd0d8e1 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:49.877537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:49.878358 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:49.878664 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:49.878898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] PATH_INFO: `/v3/projects/0a814a9a1f2548d186ddb8b8ca826698` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:49.880142 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a814a9a1f2548d186ddb8b8ca826698)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:49.882004 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:49.971637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a814a9a1f2548d186ddb8b8ca826698', 'request_id': 'req-82d56382-cda7-4b33-8fe3-829d1e55400d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:50.040686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a814a9a1f2548d186ddb8b8ca826698', 'request_id': 'req-82d56382-cda7-4b33-8fe3-829d1e55400d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:50.046426 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a814a9a1f2548d186ddb8b8ca826698', 'request_id': 'req-82d56382-cda7-4b33-8fe3-829d1e55400d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:50.057881 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] The token cache is being invalidate because project 0a814a9a1f2548d186ddb8b8ca826698 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:53:50.058169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:50.058803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-82d56382-cda7-4b33-8fe3-829d1e55400d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:50.060375 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1392/2786] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:53:49 2022] DELETE /identity/v3/projects/0a814a9a1f2548d186ddb8b8ca826698 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:50.065274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.065712 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.065849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.083339 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:50.084567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:50.141270 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1395/2787] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:50.148679 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.148969 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.149247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.167372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:50.168591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:50.176917 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1393/2788] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:50.180811 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.190412 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4744772e-0182-4282-9c2f-1c139582e65d None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.194984 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.195626 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.195937 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.196215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.196933 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:53:50.197268 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.198849 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.210229 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1396/2789] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:53:50.214291 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.223730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa01d9-ac24-4ff7-ad1e-507ac58e30ff None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.228423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.229074 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.229357 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.229630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.230334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.232286 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.316305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1394/2790] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:50.320394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.331107 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa6e63b-04a7-448e-9650-20b873437a0e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.364253 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:50.386596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.387260 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.387555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.387861 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.388555 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.390469 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.504115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1397/2791] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:50.508180 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.519599 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081d037-cd83-487e-9860-845120fd0b0d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.524331 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.524970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.525256 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.525537 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.526257 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.527893 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.539219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1395/2792] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:50.543021 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.554105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912696e8-709d-4d99-a87e-4f6879736897 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.559982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9cca26-ef6c-4129-aefc-2c871ed2531d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.559982 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cca26-ef6c-4129-aefc-2c871ed2531d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.560284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cca26-ef6c-4129-aefc-2c871ed2531d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.560509 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9cca26-ef6c-4129-aefc-2c871ed2531d admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.561243 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cca26-ef6c-4129-aefc-2c871ed2531d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.562899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9cca26-ef6c-4129-aefc-2c871ed2531d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.574691 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1398/2793] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:50.579446 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.589901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b35e50-1a57-4932-9a39-53ddc49fd29c admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.595116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.596019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.600678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.600678 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] PATH_INFO: `/v3/projects/341acea2977448f2ae5ff9d2261ad447/users/828d3a7c6aab43dd94a87ab55fe21687/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.611597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] RBAC: Authorizing `identity:create_grant(project_id=341acea2977448f2ae5ff9d2261ad447, user_id=828d3a7c6aab43dd94a87ab55fe21687, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.614267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.694116 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1396/2794] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:50 2022] PUT /identity/v3/projects/341acea2977448f2ae5ff9d2261ad447/users/828d3a7c6aab43dd94a87ab55fe21687/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:53:50.706096 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.706398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.706689 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.732576 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] MFA Rules not processed for user `828d3a7c6aab43dd94a87ab55fe21687`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:50.733804 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:50.796247 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1399/2795] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:50.800765 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.831555 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:50.862995 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075667a-ef83-44f8-bd58-5d97aa6b652f admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:50.868143 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:50.868825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:50.869112 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:50.869392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:50.870097 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:50.872058 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:50.945518 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1397/2796] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:50.949577 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:50.960876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40d868-11e7-4680-aaf3-778d8abfb34f None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.016347 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.017004 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.017284 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.017558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.018238 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.020164 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.080708 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1400/2797] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:53:50 2022] POST /identity/v3/users => generated 297 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:51.084738 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:51.095101 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32945c73-077f-460a-b61d-66aca5f04ac7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.099936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.100596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.100880 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.101160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.101911 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.103522 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.115215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1398/2798] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:53:51.119188 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:51.129777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0553af0a-8d17-43df-9756-b8b3168f9c3c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.134588 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.135744 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.136036 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.136318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] PATH_INFO: `/v3/projects/962346790310404a967b3280ca3e650a/users/9f76e9ebab574f74912db470e7b0fe63/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.150568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] RBAC: Authorizing `identity:create_grant(project_id=962346790310404a967b3280ca3e650a, user_id=9f76e9ebab574f74912db470e7b0fe63, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.153259 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.223285 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1401/2799] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:51 2022] PUT /identity/v3/projects/962346790310404a967b3280ca3e650a/users/9f76e9ebab574f74912db470e7b0fe63/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:51.227224 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:51.238581 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec414f-bd32-49f1-b252-127a935ad652 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.294686 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e5c2c3-eea1-4cff-9b52-414453c3e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.295333 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5c2c3-eea1-4cff-9b52-414453c3e1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.295615 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5c2c3-eea1-4cff-9b52-414453c3e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.295926 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e5c2c3-eea1-4cff-9b52-414453c3e1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.296648 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5c2c3-eea1-4cff-9b52-414453c3e1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.298702 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e5c2c3-eea1-4cff-9b52-414453c3e1a9 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.308250 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1399/2800] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:51 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:51.313106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:51.323573 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d5d1a-af6d-4da3-94c7-4f1d9eefd04a admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.328159 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.329054 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.329336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.329637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] PATH_INFO: `/v3/projects/962346790310404a967b3280ca3e650a/users/9f76e9ebab574f74912db470e7b0fe63/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.331646 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=962346790310404a967b3280ca3e650a, user_id=9f76e9ebab574f74912db470e7b0fe63, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.334233 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.339653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:51.344791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1402/2801] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:51 2022] PUT /identity/v3/projects/962346790310404a967b3280ca3e650a/users/9f76e9ebab574f74912db470e7b0fe63/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:51.355396 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.355713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.356025 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.382890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] MFA Rules not processed for user `9f76e9ebab574f74912db470e7b0fe63`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:51.384121 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:51.450730 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1400/2802] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:53:51.461910 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:51.463606 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:51.519783 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb479fca-e21f-42e7-8ff7-6574179105bc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.523438 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.524104 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.524394 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.524664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.533965 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.535400 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.544863 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1403/2803] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:51.639476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:51.649081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9ca4-76cb-4205-8fcf-6f9618fceea3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:51.652609 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:51.653238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:51.653523 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:51.653799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:51.673580 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:51.710416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:51.711999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:51.723618 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1401/2804] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:54.239215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:54.240737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:54.316901 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d06bcea-9e5a-4d8f-b968-00ef192f7828 service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:54.322477 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f310f6d84a3d42df9d473677ac5b6557', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:54.323125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:54.323411 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:54.323715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:54.325532 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:54.337870 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:54.339504 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a346d07-888d-46fa-a820-c856edc0ad39 service nova] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:54.350845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1404/2805] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:53:54.832171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:54.899145 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33116a5c-ab91-49c2-95c5-4efe04cd63a3 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:54.905432 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:54.907123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:54.907290 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:54.907479 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] PATH_INFO: `/v3/users/828d3a7c6aab43dd94a87ab55fe21687` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:54.908723 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=828d3a7c6aab43dd94a87ab55fe21687)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:54.910665 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:55.135108 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '828d3a7c6aab43dd94a87ab55fe21687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.135873 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '828d3a7c6aab43dd94a87ab55fe21687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.200495 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '828d3a7c6aab43dd94a87ab55fe21687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.214550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '828d3a7c6aab43dd94a87ab55fe21687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.222859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '828d3a7c6aab43dd94a87ab55fe21687'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.237489 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1402/2806] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:53:54 2022] DELETE /identity/v3/users/828d3a7c6aab43dd94a87ab55fe21687 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:55.239923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:55.240414 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:55.240519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:55.260912 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:55.262287 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:55.324554 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1405/2807] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:55.334890 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:55.336392 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:55.379324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:55.393508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e302fa-a35e-4504-ae37-abe3cee5a393 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:55.397082 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:55.397713 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:55.397993 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:55.398557 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:55.412442 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:55.414500 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:55.434703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1403/2808] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:53:55.795980 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:55.798075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:55.810483 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1822cde-236c-4a2e-be8a-2ab3508c3bc5 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:55.817047 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:55.817948 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:55.818240 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:55.818520 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] PATH_INFO: `/v3/projects/341acea2977448f2ae5ff9d2261ad447` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:55.819764 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=341acea2977448f2ae5ff9d2261ad447)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:55.822465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:55.845160 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:55.914600 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '341acea2977448f2ae5ff9d2261ad447', 'request_id': 'req-78923e95-5cfc-49de-8de6-946dfbd03fb5'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.965645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '341acea2977448f2ae5ff9d2261ad447', 'request_id': 'req-78923e95-5cfc-49de-8de6-946dfbd03fb5'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.971944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '341acea2977448f2ae5ff9d2261ad447', 'request_id': 'req-78923e95-5cfc-49de-8de6-946dfbd03fb5'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.983051 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] The token cache is being invalidate because project 341acea2977448f2ae5ff9d2261ad447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:53:55.983335 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.983973 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:55.986083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1406/2809] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:53:55 2022] DELETE /identity/v3/projects/341acea2977448f2ae5ff9d2261ad447 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:55.990026 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:55.999820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055be6b-d3b3-4e87-983c-941df6a5fde2 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:56.055091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:56.055979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:56.056264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:56.056542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] PATH_INFO: `/v3/users/9f76e9ebab574f74912db470e7b0fe63` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:56.057720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f76e9ebab574f74912db470e7b0fe63)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:56.059629 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:56.206743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f76e9ebab574f74912db470e7b0fe63'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.207535 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f76e9ebab574f74912db470e7b0fe63'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.297850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f76e9ebab574f74912db470e7b0fe63'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.310455 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f76e9ebab574f74912db470e7b0fe63'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.317162 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-2b913bfa-4f3b-47d4-8b3a-840ad9e5c175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f76e9ebab574f74912db470e7b0fe63'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.323078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1404/2810] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:53:55 2022] DELETE /identity/v3/users/9f76e9ebab574f74912db470e7b0fe63 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:56.698895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:56.700332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:56.714762 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78923e95-5cfc-49de-8de6-946dfbd03fb5 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:56.771737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:56.772507 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:56.772787 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:56.773059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] PATH_INFO: `/v3/projects/962346790310404a967b3280ca3e650a` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:56.774220 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=962346790310404a967b3280ca3e650a)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:56.776158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:56.863065 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '962346790310404a967b3280ca3e650a', 'request_id': 'req-e27b3373-4da8-40fd-b587-5f5218898dfc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.919489 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '962346790310404a967b3280ca3e650a', 'request_id': 'req-e27b3373-4da8-40fd-b587-5f5218898dfc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.925215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '962346790310404a967b3280ca3e650a', 'request_id': 'req-e27b3373-4da8-40fd-b587-5f5218898dfc'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.936189 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] The token cache is being invalidate because project 962346790310404a967b3280ca3e650a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80392) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:53:56.936473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.937100 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-e27b3373-4da8-40fd-b587-5f5218898dfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:53:56.939172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1407/2811] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:53:56 2022] DELETE /identity/v3/projects/962346790310404a967b3280ca3e650a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:56.944383 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:56.944677 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:56.944958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:56.962642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:56.963848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:57.024771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1405/2812] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:57.032539 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.032964 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.033310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.051567 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:57.052866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:57.060656 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1408/2813] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:57.065311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.070065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:57.075215 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e3119a-8716-48d2-aff2-7c4c60382467 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.079799 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.082313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.082313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.082313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.082313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80393) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:53:57.082313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.083709 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.094923 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1406/2814] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:53:57.099080 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.108201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6169c960-a0af-44bf-a9a9-a156e1d70cb9 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.113283 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.113917 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.114199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.114472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.115157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.117067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.213605 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1409/2815] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:53:57 2022] POST /identity/v3/projects => generated 377 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:53:57.229282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.229282 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a514f-c47b-42b1-a48f-be0120f3adf7 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.285055 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.285636 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.285925 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.286204 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.286898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.288867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.366760 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1407/2816] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:53:57 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:53:57.371212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.384492 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afcae24-c3c9-4654-b436-77c768a5fc64 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.389542 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.390227 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.390513 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.390794 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.391519 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.393192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.398558 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:57.404767 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1410/2817] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:57.410630 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.420812 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda64736-cc20-4857-b3e4-83f11f5efe98 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.426069 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efa340-c8a5-4f34-9584-7dbd1b37848b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.429699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa340-c8a5-4f34-9584-7dbd1b37848b admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.429699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa340-c8a5-4f34-9584-7dbd1b37848b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.429699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efa340-c8a5-4f34-9584-7dbd1b37848b admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.429699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efa340-c8a5-4f34-9584-7dbd1b37848b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.429699 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efa340-c8a5-4f34-9584-7dbd1b37848b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.439719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1408/2818] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:53:57.443777 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.453675 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462846b-f60d-4600-b788-e17c10481be5 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.458388 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.459248 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.459534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.459847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] PATH_INFO: `/v3/projects/0ceb24abde494e559c3d855d24025e50/users/00b05c9aa4ee46dfbc93b1b106f626a8/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.475583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ceb24abde494e559c3d855d24025e50, user_id=00b05c9aa4ee46dfbc93b1b106f626a8, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.478295 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.598033 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1411/2819] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:57 2022] PUT /identity/v3/projects/0ceb24abde494e559c3d855d24025e50/users/00b05c9aa4ee46dfbc93b1b106f626a8/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:57.612856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.613141 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.613416 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.644755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] MFA Rules not processed for user `00b05c9aa4ee46dfbc93b1b106f626a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:57.645974 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:57.707514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1409/2820] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:57.712210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.773550 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb927c6-c53f-46fd-8adf-307606fd623e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.778122 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.778823 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.779111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.779385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.780157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.782679 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:57.870183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1412/2821] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:53:57 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:57.874255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:57.884639 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fa2a6-03fb-44b8-a792-33609020fc7b None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:57.935329 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:57.941011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:57.941663 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:57.941942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:57.951943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:57.951943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:57.951943 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.046263 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1410/2822] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:53:57 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:58.050142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.060493 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aff602-2f33-418d-829c-ca3d66f4bb53 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.065116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.065763 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.066046 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.066324 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.067044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.068685 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.079418 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1413/2823] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:58.083138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.092913 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4357b41a-04f2-4bd3-8688-e6aba7a00347 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.098087 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be521be9-6f42-4f64-ac6a-b0ba1f9cfe4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.098733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be521be9-6f42-4f64-ac6a-b0ba1f9cfe4a admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.099017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be521be9-6f42-4f64-ac6a-b0ba1f9cfe4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.099295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be521be9-6f42-4f64-ac6a-b0ba1f9cfe4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.100035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be521be9-6f42-4f64-ac6a-b0ba1f9cfe4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.101645 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be521be9-6f42-4f64-ac6a-b0ba1f9cfe4a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.112089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1411/2824] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:58.116003 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.125575 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed04b8cf-b711-4379-a37a-378cafd3a3cc admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.130066 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.130920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.131202 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.131475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] PATH_INFO: `/v3/projects/3e1e65631cb54aec9eb83233de4baabc/users/c7c1f904e52349d2befa02d9486e9179/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.146090 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1e65631cb54aec9eb83233de4baabc, user_id=c7c1f904e52349d2befa02d9486e9179, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.148731 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.229737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1414/2825] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:58 2022] PUT /identity/v3/projects/3e1e65631cb54aec9eb83233de4baabc/users/c7c1f904e52349d2befa02d9486e9179/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:53:58.239800 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.240091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.240376 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.267779 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] MFA Rules not processed for user `c7c1f904e52349d2befa02d9486e9179`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:58.268979 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:58.331037 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1412/2826] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:58.335510 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.402201 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2d692-c1ec-45d1-bd6c-78571fa16b1e admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.406747 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.407393 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.407703 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.407978 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.408649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.410579 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.498756 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1415/2827] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:53:58 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:58.503355 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.514105 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54088ccd-782c-4ec4-9cae-23ce219f411e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.569876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.570536 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.570818 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.571090 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.571825 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.573708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.651427 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1413/2828] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:53:58 2022] POST /identity/v3/users => generated 295 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:53:58.655424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.665695 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08516675-4712-478a-9c27-edccb1ad7a00 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.671334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.671334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.671334 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.672214 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.672760 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.674357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.679598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:58.686106 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1416/2829] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:58.690667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.700550 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8201c0-c1e4-4c11-97a8-d1463dd436f5 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.705264 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.706138 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.706418 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.706690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] PATH_INFO: `/v3/projects/7bc0d3bed3fd49b1909b0521886d11db/users/b87efdaa6610425b957c8e9f179ff3b9/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.721597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc0d3bed3fd49b1909b0521886d11db, user_id=b87efdaa6610425b957c8e9f179ff3b9, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.724281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.796468 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1414/2830] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:58 2022] PUT /identity/v3/projects/7bc0d3bed3fd49b1909b0521886d11db/users/b87efdaa6610425b957c8e9f179ff3b9/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:58.801022 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.813298 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bc4ff-92bc-4575-a2c6-19e4dc270a47 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.868392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0fd47-2712-4c96-972a-b21b645f82a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.869052 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0fd47-2712-4c96-972a-b21b645f82a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.869336 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0fd47-2712-4c96-972a-b21b645f82a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.869612 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0fd47-2712-4c96-972a-b21b645f82a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.870511 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0fd47-2712-4c96-972a-b21b645f82a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.872148 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0fd47-2712-4c96-972a-b21b645f82a6 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.882472 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1417/2831] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:53:58.886833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:58.896940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f08bd-99a6-45ae-a384-04c948e32401 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:58.901621 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:58.902499 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.902784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.903061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] PATH_INFO: `/v3/projects/7bc0d3bed3fd49b1909b0521886d11db/users/b87efdaa6610425b957c8e9f179ff3b9/roles/6c6c8f2c4f144911a9a411b476178fc4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.905042 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc0d3bed3fd49b1909b0521886d11db, user_id=b87efdaa6610425b957c8e9f179ff3b9, role_id=6c6c8f2c4f144911a9a411b476178fc4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:58.908352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:58.918743 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1415/2832] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:53:58 2022] PUT /identity/v3/projects/7bc0d3bed3fd49b1909b0521886d11db/users/b87efdaa6610425b957c8e9f179ff3b9/roles/6c6c8f2c4f144911a9a411b476178fc4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:53:58.930216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:58.930523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:58.930807 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:58.956218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] MFA Rules not processed for user `b87efdaa6610425b957c8e9f179ff3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:53:58.957415 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:59.026157 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1418/2833] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:53:59.038086 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:53:59.039824 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:53:59.060642 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:53:59.097637 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d4bf9-d972-44e7-b625-83a42b32dc5b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:53:59.101352 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:53:59.102029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:53:59.102323 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:53:59.102592 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:53:59.158746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:53:59.160146 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:53:59.169546 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1416/2834] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:53:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:01.087179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:01.088613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:01.100782 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c1451-1a6b-4dff-bfa4-5da46bf12ee7 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:01.104222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:01.104846 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:01.105127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:01.105402 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:01.160541 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:01.162005 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:01.170995 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1419/2835] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:54:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:02.400716 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:02.409188 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb83b165-48ce-4a57-9db3-d9f231ba85fe service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:02.412781 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c0f9e-645e-41f8-9e00-fb756164192f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:02.413404 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0f9e-645e-41f8-9e00-fb756164192f service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:02.413684 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0f9e-645e-41f8-9e00-fb756164192f service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:02.413959 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c0f9e-645e-41f8-9e00-fb756164192f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:02.423213 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c0f9e-645e-41f8-9e00-fb756164192f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:02.424656 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c0f9e-645e-41f8-9e00-fb756164192f service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:02.433888 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1417/2836] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:54:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:06.880922 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:06.901187 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=100 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:06.942363 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f7b2e-ffb6-4b4f-a294-0d2a1a0f38ca service cinder] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:06.946888 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:06.947775 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:06.948059 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:06.948332 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] PATH_INFO: `/v3/users/00b05c9aa4ee46dfbc93b1b106f626a8` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:06.949501 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b05c9aa4ee46dfbc93b1b106f626a8)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:06.951406 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:07.116788 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b05c9aa4ee46dfbc93b1b106f626a8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.117364 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b05c9aa4ee46dfbc93b1b106f626a8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.194068 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b05c9aa4ee46dfbc93b1b106f626a8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.201099 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b05c9aa4ee46dfbc93b1b106f626a8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.212829 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b05c9aa4ee46dfbc93b1b106f626a8'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.219960 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1420/2837] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:54:06 2022] DELETE /identity/v3/users/00b05c9aa4ee46dfbc93b1b106f626a8 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:54:07.224505 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:07.224977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:07.225109 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:07.242887 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:07.244132 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:07.305190 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1418/2838] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:07.314979 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:07.316453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:07.374508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a34a4-2f36-48f1-9627-0c3d8151e671 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:07.378222 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:07.378868 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:07.379156 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:07.379433 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:07.388734 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:07.390172 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:07.399586 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1421/2839] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:07.768419 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:07.769848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:07.782670 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89211039-c8e6-4261-84a9-2549d181dc33 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:07.787441 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:07.788318 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:07.788597 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:07.788867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] PATH_INFO: `/v3/projects/0ceb24abde494e559c3d855d24025e50` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:07.790652 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ceb24abde494e559c3d855d24025e50)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:07.791964 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:07.892701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ceb24abde494e559c3d855d24025e50', 'request_id': 'req-f2b9d61c-955f-4b95-8849-ab95a40e721d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.896391 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:07.948763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ceb24abde494e559c3d855d24025e50', 'request_id': 'req-f2b9d61c-955f-4b95-8849-ab95a40e721d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.954884 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ceb24abde494e559c3d855d24025e50', 'request_id': 'req-f2b9d61c-955f-4b95-8849-ab95a40e721d'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.968118 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] The token cache is being invalidate because project 0ceb24abde494e559c3d855d24025e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:54:07.968402 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.973477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:07.973477 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1419/2840] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:54:07 2022] DELETE /identity/v3/projects/0ceb24abde494e559c3d855d24025e50 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:54:07.974809 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:07.984387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c902d-2ccd-4088-b918-85f249f2bc43 service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:08.025987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:08.041568 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:08.042434 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:08.042715 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:08.042990 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] PATH_INFO: `/v3/users/c7c1f904e52349d2befa02d9486e9179` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:08.044186 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7c1f904e52349d2befa02d9486e9179)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:08.046137 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:08.184035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7c1f904e52349d2befa02d9486e9179'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.184599 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7c1f904e52349d2befa02d9486e9179'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.234485 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7c1f904e52349d2befa02d9486e9179'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.242697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7c1f904e52349d2befa02d9486e9179'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.255266 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7c1f904e52349d2befa02d9486e9179'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.262194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1422/2841] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:54:07 2022] DELETE /identity/v3/users/c7c1f904e52349d2befa02d9486e9179 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:54:08.652970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:08.654567 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:08.670413 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9d61c-955f-4b95-8849-ab95a40e721d admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:08.729022 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:08.729878 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:08.730171 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:08.730434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] PATH_INFO: `/v3/projects/3e1e65631cb54aec9eb83233de4baabc` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:08.731617 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1e65631cb54aec9eb83233de4baabc)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:08.733542 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:08.829241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1e65631cb54aec9eb83233de4baabc', 'request_id': 'req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.891712 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1e65631cb54aec9eb83233de4baabc', 'request_id': 'req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.897787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1e65631cb54aec9eb83233de4baabc', 'request_id': 'req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.908936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] The token cache is being invalidate because project 3e1e65631cb54aec9eb83233de4baabc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:54:08.909308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.910372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:08.911942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1420/2842] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:54:08 2022] DELETE /identity/v3/projects/3e1e65631cb54aec9eb83233de4baabc => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 06 20:54:08.915918 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:08.917315 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:08.930464 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd3a9e5-e040-4d50-9953-2d7b55ea0311 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:08.987200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:08.988387 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:08.988684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:08.988953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] PATH_INFO: `/v3/users/b87efdaa6610425b957c8e9f179ff3b9` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:08.990239 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87efdaa6610425b957c8e9f179ff3b9)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:08.992226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:09.128027 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87efdaa6610425b957c8e9f179ff3b9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.128670 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87efdaa6610425b957c8e9f179ff3b9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.171557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87efdaa6610425b957c8e9f179ff3b9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.178610 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87efdaa6610425b957c8e9f179ff3b9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.179797 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:09.190981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-8c356597-b4c8-4782-a765-f7cce57567db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87efdaa6610425b957c8e9f179ff3b9'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.196872 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1423/2843] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:54:08 2022] DELETE /identity/v3/users/b87efdaa6610425b957c8e9f179ff3b9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:54:09.624595 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:09.626013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:09.639399 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6eb129-83e3-4af5-8f1f-7160ee9d67a4 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:09.698199 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:09.699052 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:09.699332 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:09.699606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] PATH_INFO: `/v3/projects/7bc0d3bed3fd49b1909b0521886d11db` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:09.700933 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc0d3bed3fd49b1909b0521886d11db)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:09.702674 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:09.717719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:09.811876 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc0d3bed3fd49b1909b0521886d11db', 'request_id': 'req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.886156 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc0d3bed3fd49b1909b0521886d11db', 'request_id': 'req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.891805 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc0d3bed3fd49b1909b0521886d11db', 'request_id': 'req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.902871 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] The token cache is being invalidate because project 7bc0d3bed3fd49b1909b0521886d11db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:54:09.903157 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.903848 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-0e5d959a-fa4e-4340-b9c5-376ff12c34f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:54:09.905492 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1421/2844] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:54:09 2022] DELETE /identity/v3/projects/7bc0d3bed3fd49b1909b0521886d11db => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:54:09.910500 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:09.910789 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:09.911098 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:09.930515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:09.931768 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:10.121578 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1424/2845] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:10.129297 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.129583 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.129859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.147885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:10.149008 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:10.156771 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1422/2846] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:10.161376 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.171105 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a581b-36b8-488e-9685-1844269c5c65 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.176737 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.177374 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.177653 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.177953 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.178660 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:54:10.178957 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.180564 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.191902 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1425/2847] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:54:10.196029 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.205232 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d10b80-60f5-4388-b35d-9f4c398d3ef3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.209875 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.210508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.210790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.211063 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.211784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.213657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.288010 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1423/2848] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:54:10 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:54:10.292194 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.302607 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc08bb-cd14-4c4a-996f-675651f0e09f admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.357223 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.357876 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.358162 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.358557 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.359242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.361292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.440416 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1426/2849] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:54:10 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:54:10.444233 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.454690 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948cc2b-676c-40d4-a389-948ede09774b admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.459790 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.460434 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.460720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.460997 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.461708 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.463313 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.474372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1424/2850] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:54:10.478138 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.488083 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36034e9f-3cbb-4b48-9186-ba7e51468011 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.492661 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4940072f-95f3-4994-bc73-fb19ba51fafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.493303 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940072f-95f3-4994-bc73-fb19ba51fafb admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.493584 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940072f-95f3-4994-bc73-fb19ba51fafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.493860 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940072f-95f3-4994-bc73-fb19ba51fafb admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.494581 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940072f-95f3-4994-bc73-fb19ba51fafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.496192 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4940072f-95f3-4994-bc73-fb19ba51fafb admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.505855 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1427/2851] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:54:10.510011 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.520324 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fdfc9-89cd-4d6c-b2fb-3b3f2cd89176 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.524934 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.525802 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.526089 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.526369 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] PATH_INFO: `/v3/projects/615b738dad2c43d8a7cef6086bb467de/users/ea091ae58b114083abd29801f6d8baff/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.541061 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615b738dad2c43d8a7cef6086bb467de, user_id=ea091ae58b114083abd29801f6d8baff, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.543735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.600449 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1425/2852] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:54:10 2022] PUT /identity/v3/projects/615b738dad2c43d8a7cef6086bb467de/users/ea091ae58b114083abd29801f6d8baff/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:54:10.609278 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855321a0-761c-4b88-b67f-e2a34aee9ad3 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.609571 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855321a0-761c-4b88-b67f-e2a34aee9ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.609847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855321a0-761c-4b88-b67f-e2a34aee9ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.626881 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-855321a0-761c-4b88-b67f-e2a34aee9ad3 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:10.628109 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-855321a0-761c-4b88-b67f-e2a34aee9ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:10.668746 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:10.685453 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1428/2853] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:10.697091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:10.697091 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:10.752193 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52714b7-2968-47df-8151-5c70ab330f50 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:10.755951 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:10.756578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:10.756859 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:10.757134 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:10.766186 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:10.767589 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:10.777175 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1426/2854] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:11.898169 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c198a96-5968-4a50-bd7c-167c247feb3b None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:11.898724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c198a96-5968-4a50-bd7c-167c247feb3b None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:11.898724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c198a96-5968-4a50-bd7c-167c247feb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:11.926074 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-3c198a96-5968-4a50-bd7c-167c247feb3b None None] MFA Rules not processed for user `ea091ae58b114083abd29801f6d8baff`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:11.927286 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-3c198a96-5968-4a50-bd7c-167c247feb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:11.992671 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1429/2855] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:12.003514 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:12.004855 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:12.050940 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:12.062409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39889aa9-8c68-428f-93a5-5ee883847e06 service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:12.066030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:12.066697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:12.066980 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:12.067260 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:12.075885 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:12.077350 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:12.086774 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1427/2856] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:54:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:13.718508 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adf1ad-6d92-42ed-a842-7e16b89db087 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:13.719031 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adf1ad-6d92-42ed-a842-7e16b89db087 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:13.719153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83adf1ad-6d92-42ed-a842-7e16b89db087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:13.735944 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-83adf1ad-6d92-42ed-a842-7e16b89db087 None None] MFA Rules not processed for user `ea091ae58b114083abd29801f6d8baff`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:13.737121 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-83adf1ad-6d92-42ed-a842-7e16b89db087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:13.744274 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1430/2857] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:54:13.971961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:13.974017 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:13.986371 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af35a9-b6cf-4a5d-9e2b-6b47e4b27759 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:13.990775 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:13.991414 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:13.991737 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:13.992015 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:13.993801 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:14.003114 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:14.004667 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b285043f-f1b0-4461-bdbc-179da18f5a5b service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:14.012729 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1428/2858] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:54:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:54:23.730793 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=41 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:24.008942 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=13 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:30.642684 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:30.643158 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:30.643265 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:30.670317 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] MFA Rules not processed for user `ea091ae58b114083abd29801f6d8baff`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:30.671506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:30.693834 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1431/2859] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:54:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:40.691890 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=15 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:54:51.176515 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cf451-a8ce-4e25-855b-7aecedd69a84 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:51.177024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cf451-a8ce-4e25-855b-7aecedd69a84 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:51.177144 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cf451-a8ce-4e25-855b-7aecedd69a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:51.205255 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a98cf451-a8ce-4e25-855b-7aecedd69a84 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:54:51.206439 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a98cf451-a8ce-4e25-855b-7aecedd69a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:51.283476 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1429/2860] 10.222.0.41 () {58 vars in 1009 bytes} [Mon Jun 6 20:54:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:54:51.449409 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:54:51.450875 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:54:51.525791 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d46ba7-f6f6-4ab0-9baa-6f8f90bcc606 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:54:51.531473 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:54:51.532150 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:54:51.532458 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:54:51.532732 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:54:51.541061 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:54:51.543215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:54:51.551385 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1432/2861] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:54:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:55:01.277124 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=39 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:01.547969 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=35 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:07.940065 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:07.940554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:07.940554 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:07.958814 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] MFA Rules not processed for user `9196a10f723649839a7eca20e99d84f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:07.960041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:07.967161 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1430/2862] 10.222.0.41 () {58 vars in 967 bytes} [Mon Jun 6 20:55:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:07.978997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:07.989407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8efab-55b8-443a-ba3e-0aed55f99f3a service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:07.994424 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e168d14c0eac4ee2a5acdc23c39118d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:07.995044 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:07.995331 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:07.995609 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:08.004043 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:08.005604 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:08.014371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1433/2863] 10.222.0.41 () {60 vars in 1384 bytes} [Mon Jun 6 20:55:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 06 20:55:11.039867 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:11.041345 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:11.111600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263932c-715e-498d-bc83-46cfe35938e3 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:11.116572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:11.117214 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:11.117538 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:11.117816 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:11.119572 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:11.143830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:11.143830 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:11.144012 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:11.150005 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1431/2864] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:55:11.168515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb03c1-04d1-4646-9ad2-121c2797467d service swift] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:11.173069 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:11.173926 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:11.174465 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:11.174698 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] PATH_INFO: `/v3/users/ea091ae58b114083abd29801f6d8baff` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:11.175895 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea091ae58b114083abd29801f6d8baff)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:11.177780 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:11.315445 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea091ae58b114083abd29801f6d8baff'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.316127 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea091ae58b114083abd29801f6d8baff'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.359149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea091ae58b114083abd29801f6d8baff'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.366359 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea091ae58b114083abd29801f6d8baff'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.378657 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-bd60289b-a75c-48bd-aceb-c0fa1e49256d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea091ae58b114083abd29801f6d8baff'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.384847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1434/2865] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:55:11 2022] DELETE /identity/v3/users/ea091ae58b114083abd29801f6d8baff => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:11.614019 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:11.625433 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617acd1c-0cfd-462c-b8ec-683ca1bb4e4a service placement] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:11.691140 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:11.692038 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:11.692321 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:11.692596 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] PATH_INFO: `/v3/projects/615b738dad2c43d8a7cef6086bb467de` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:11.693798 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=615b738dad2c43d8a7cef6086bb467de)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:11.695764 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:11.795294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615b738dad2c43d8a7cef6086bb467de', 'request_id': 'req-f2e6de12-64b9-4cc1-be08-24bae8759851'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.885241 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615b738dad2c43d8a7cef6086bb467de', 'request_id': 'req-f2e6de12-64b9-4cc1-be08-24bae8759851'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.891784 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615b738dad2c43d8a7cef6086bb467de', 'request_id': 'req-f2e6de12-64b9-4cc1-be08-24bae8759851'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.915344 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] The token cache is being invalidate because project 615b738dad2c43d8a7cef6086bb467de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:55:11.915632 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.917657 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-f2e6de12-64b9-4cc1-be08-24bae8759851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:11.917892 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1432/2866] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:55:11 2022] DELETE /identity/v3/projects/615b738dad2c43d8a7cef6086bb467de => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:11.922704 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:11.923125 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:11.923455 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:11.978583 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:11.981517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:12.037894 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1435/2867] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:12.046558 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.047041 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.047160 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.053311 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:12.065906 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:12.067093 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:12.075002 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1433/2868] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:12.079079 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.088475 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3512c1-7420-47c3-9263-b1f0ad2719c2 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.092930 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.093566 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.093847 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.094120 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.094833 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:55:12.095131 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.096843 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.105055 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:12.108814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1436/2869] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:55:12.112987 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.122408 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e4b93-dd42-4db7-871c-39bdf8ab6149 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.127705 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.128348 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.128626 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.128898 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.129578 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.131444 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.246450 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1434/2870] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:55:12 2022] POST /identity/v3/projects => generated 365 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:55:12.251081 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.261153 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736894e-68cd-46a1-a91c-100ad033ce2c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.315989 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.316662 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.316949 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.317230 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.317997 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.319923 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.416226 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1437/2871] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:55:12 2022] POST /identity/v3/users => generated 283 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:55:12.421895 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.432372 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910170a-8856-4fbc-91a2-b692f30a8ab2 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.437035 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.437949 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.438229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.438501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.439229 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.440961 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.450850 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1435/2872] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:55:12.455370 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.465517 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfca89d-d723-492d-be89-00ecededfb9c admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.470905 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a0073-b567-4fa2-813f-3fe9851589ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.473373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0073-b567-4fa2-813f-3fe9851589ea admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.473373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0073-b567-4fa2-813f-3fe9851589ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.473373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a0073-b567-4fa2-813f-3fe9851589ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.473373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a0073-b567-4fa2-813f-3fe9851589ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.474680 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a0073-b567-4fa2-813f-3fe9851589ea admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.485992 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1438/2873] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:55:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:55:12.490411 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.500635 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a87aa-9b62-4e34-896d-56be7866d831 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.505247 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.506183 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.506460 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.506733 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] PATH_INFO: `/v3/projects/e119786451714b04abde2cc0375c7de4/users/d86f7829d8c844ad8f4afc1ad30bf847/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.522261 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e119786451714b04abde2cc0375c7de4, user_id=d86f7829d8c844ad8f4afc1ad30bf847, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.525501 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.593809 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1436/2874] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:55:12 2022] PUT /identity/v3/projects/e119786451714b04abde2cc0375c7de4/users/d86f7829d8c844ad8f4afc1ad30bf847/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:12.603832 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.604126 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.604408 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.631115 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] MFA Rules not processed for user `d86f7829d8c844ad8f4afc1ad30bf847`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:12.632350 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:12.694011 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1439/2875] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:12.705012 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:12.706703 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:12.782843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177961b6-91a2-44f3-a838-c7235693ff00 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:12.786425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:12.787056 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:12.787340 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:12.787620 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:12.795688 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:12.797123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:12.805697 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1437/2876] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:55:22.690976 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:22.801788 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:25.081946 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:25.083328 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:25.171033 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035c4bb-4d8e-4c6a-b05a-009885b331b4 None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:25.176091 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:25.176729 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:25.177034 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:25.177310 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:25.179111 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:25.190866 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:25.192457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:25.202205 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1440/2877] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:55:35.198002 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=41 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:40.627728 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:40.629842 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:40.707591 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ebdde-5d65-4485-b5b6-8f7017c57755 service cinder] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:40.711405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:40.712077 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:40.712359 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:40.712631 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:40.714429 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:40.777968 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:40.779398 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:40.789480 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1438/2878] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:55:47.907673 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:47.909319 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:47.924181 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5c0cb-d4d2-4863-8d40-2e26f8ce3de0 service placement] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:47.929812 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:47.930727 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:47.931008 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:47.931280 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] PATH_INFO: `/v3/users/2bb3e3fc9a494cdf8abd264038af044c` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:47.932693 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bb3e3fc9a494cdf8abd264038af044c)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:47.934697 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:48.097129 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bb3e3fc9a494cdf8abd264038af044c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.097814 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bb3e3fc9a494cdf8abd264038af044c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.169360 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bb3e3fc9a494cdf8abd264038af044c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.177014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bb3e3fc9a494cdf8abd264038af044c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.188994 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bb3e3fc9a494cdf8abd264038af044c'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.195523 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1441/2879] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:55:47 2022] DELETE /identity/v3/users/2bb3e3fc9a494cdf8abd264038af044c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:48.468847 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:48.481579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3982d-4f5e-4c7f-a277-65e2a73b320f service neutron] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:48.539622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:48.540423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:48.540700 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:48.540977 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] PATH_INFO: `/v3/projects/c1ea53cfd795419cacebe7f2d071f37f` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:48.542293 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ea53cfd795419cacebe7f2d071f37f)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:48.544284 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:48.644931 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:48.656701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ea53cfd795419cacebe7f2d071f37f', 'request_id': 'req-d1d584a2-de0d-452b-b916-d7eaef3b9d71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.752833 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ea53cfd795419cacebe7f2d071f37f', 'request_id': 'req-d1d584a2-de0d-452b-b916-d7eaef3b9d71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.759236 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ea53cfd795419cacebe7f2d071f37f', 'request_id': 'req-d1d584a2-de0d-452b-b916-d7eaef3b9d71'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.770719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] The token cache is being invalidate because project c1ea53cfd795419cacebe7f2d071f37f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:55:48.771013 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.771649 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:48.773281 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1439/2880] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:55:48 2022] DELETE /identity/v3/projects/c1ea53cfd795419cacebe7f2d071f37f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:48.777242 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:48.778713 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:48.792748 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe645ab-877b-4729-995d-9bd9973b29d4 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:48.849292 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:48.864405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:48.864405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:48.864405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] PATH_INFO: `/v3/users/610a04ce87fc4dc4b103811207caaa57` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:48.864405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=610a04ce87fc4dc4b103811207caaa57)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:48.864405 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:49.074758 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '610a04ce87fc4dc4b103811207caaa57'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.075398 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '610a04ce87fc4dc4b103811207caaa57'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.146436 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '610a04ce87fc4dc4b103811207caaa57'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.153645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '610a04ce87fc4dc4b103811207caaa57'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.161272 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:49.165920 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '610a04ce87fc4dc4b103811207caaa57'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.172062 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1442/2881] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:55:48 2022] DELETE /identity/v3/users/610a04ce87fc4dc4b103811207caaa57 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:49.410918 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:49.412485 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:49.425986 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d584a2-de0d-452b-b916-d7eaef3b9d71 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:49.615646 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:49.616423 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:49.616701 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:49.616970 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] PATH_INFO: `/v3/projects/27eabacaf0b14d5ba6f17560f8f4c762` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:49.618123 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27eabacaf0b14d5ba6f17560f8f4c762)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:49.620018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:49.728078 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27eabacaf0b14d5ba6f17560f8f4c762', 'request_id': 'req-efe068b3-32a9-4825-b01a-a75093c00ec5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.815060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27eabacaf0b14d5ba6f17560f8f4c762', 'request_id': 'req-efe068b3-32a9-4825-b01a-a75093c00ec5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.820605 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27eabacaf0b14d5ba6f17560f8f4c762', 'request_id': 'req-efe068b3-32a9-4825-b01a-a75093c00ec5'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.831209 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] The token cache is being invalidate because project 27eabacaf0b14d5ba6f17560f8f4c762 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:55:49.831487 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.832128 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-efe068b3-32a9-4825-b01a-a75093c00ec5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:49.833622 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1440/2882] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:55:49 2022] DELETE /identity/v3/projects/27eabacaf0b14d5ba6f17560f8f4c762 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:51.203534 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:51.204999 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:51.217463 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ec18b-9929-4bd3-9b31-0efbe00a6960 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:51.273482 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:51.274357 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:51.274642 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:51.274927 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] PATH_INFO: `/v3/users/d86f7829d8c844ad8f4afc1ad30bf847` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:51.276149 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d86f7829d8c844ad8f4afc1ad30bf847)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:51.278058 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:51.450896 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd86f7829d8c844ad8f4afc1ad30bf847'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:51.451457 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd86f7829d8c844ad8f4afc1ad30bf847'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:51.512527 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd86f7829d8c844ad8f4afc1ad30bf847'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:51.519914 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd86f7829d8c844ad8f4afc1ad30bf847'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:51.531506 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd86f7829d8c844ad8f4afc1ad30bf847'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:51.538097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1443/2883] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:55:51 2022] DELETE /identity/v3/users/d86f7829d8c844ad8f4afc1ad30bf847 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:51.542820 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:51.543294 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:51.543457 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:51.561060 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:51.562252 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:51.605027 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:51.619018 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1441/2884] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:51.628723 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:51.630183 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:51.686916 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238ec3-3a87-41df-86a4-623557c6371d admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:51.691116 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eada1-4ed7-4af8-b0b5-5c9ad69ad524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:51.691776 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eada1-4ed7-4af8-b0b5-5c9ad69ad524 service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:51.692067 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eada1-4ed7-4af8-b0b5-5c9ad69ad524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:51.692340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eada1-4ed7-4af8-b0b5-5c9ad69ad524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:51.701522 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eada1-4ed7-4af8-b0b5-5c9ad69ad524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:51.702786 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eada1-4ed7-4af8-b0b5-5c9ad69ad524 service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:51.712725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1444/2885] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 06 20:55:52.113587 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.115030 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:52.127194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a408dad6-a970-420a-a070-f2096668f2fc None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.132603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.133755 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.134081 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.134405 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] PATH_INFO: `/v3/projects/e119786451714b04abde2cc0375c7de4` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.136660 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=e119786451714b04abde2cc0375c7de4)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.138608 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:52.221508 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e119786451714b04abde2cc0375c7de4', 'request_id': 'req-10cbd247-d618-451a-bdd1-5459ed983aea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:52.295445 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e119786451714b04abde2cc0375c7de4', 'request_id': 'req-10cbd247-d618-451a-bdd1-5459ed983aea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:52.301425 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e119786451714b04abde2cc0375c7de4', 'request_id': 'req-10cbd247-d618-451a-bdd1-5459ed983aea'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:52.312319 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] The token cache is being invalidate because project e119786451714b04abde2cc0375c7de4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:55:52.312606 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:52.313235 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-10cbd247-d618-451a-bdd1-5459ed983aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:52.314763 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1442/2886] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:55:52 2022] DELETE /identity/v3/projects/e119786451714b04abde2cc0375c7de4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:52.320167 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.320648 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.320751 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.338848 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:52.340218 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:52.348441 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:52.399633 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1445/2887] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:52.408271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.408569 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.408849 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.427363 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:52.428579 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:52.436243 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1443/2888] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:52.440344 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.450649 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54327058-cd18-41ba-a8bf-be97e19868fe None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.455210 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.455886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.456199 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.456474 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.457215 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80392) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 06 20:55:52.457515 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.459103 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:52.471191 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1446/2889] 10.222.0.41 () {58 vars in 1152 bytes} [Mon Jun 6 20:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 06 20:55:52.475600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.484603 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8be61-f1eb-4452-9c80-2653621ade20 None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.489719 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.490349 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.490628 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.490901 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.494295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.494295 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:52.611958 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1444/2890] 10.222.0.41 () {60 vars in 1152 bytes} [Mon Jun 6 20:55:52 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 06 20:55:52.616407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.626142 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b22b8d-d119-4cb2-b784-f45775500a26 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.682714 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.683371 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.683655 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.683981 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] PATH_INFO: `/v3/users` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.684828 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.686724 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:52.792407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1447/2891] 10.222.0.41 () {60 vars in 1143 bytes} [Mon Jun 6 20:55:52 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 06 20:55:52.796532 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.807494 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f3890-b235-4206-89d4-c791c6a6f7cf admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.812080 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.812720 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.812999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.813271 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] PATH_INFO: `/v3/roles` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.814050 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.815680 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:52.825803 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1445/2892] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 06 20:55:52.829620 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.839407 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ace51f9-b755-45b5-b6b1-4930c1a96436 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.843908 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281a10f-0484-4cf3-aa54-4d2237ed0e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.844544 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281a10f-0484-4cf3-aa54-4d2237ed0e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.844826 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281a10f-0484-4cf3-aa54-4d2237ed0e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.845097 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281a10f-0484-4cf3-aa54-4d2237ed0e0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.845808 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281a10f-0484-4cf3-aa54-4d2237ed0e0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.848028 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281a10f-0484-4cf3-aa54-4d2237ed0e0d admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:52.858088 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1448/2893] 10.222.0.41 () {58 vars in 1121 bytes} [Mon Jun 6 20:55:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 06 20:55:52.862028 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:52.872328 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3643-5592-4b57-9b97-d2dfb7e32759 admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:52.876882 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:52.877746 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] REQUEST_METHOD: `PUT` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:52.878032 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:52.878308 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] PATH_INFO: `/v3/projects/f4fef0a4c2b740d3beec3b0acb07a7f2/users/7170ab9fc27f4ff694bc2ebe056d54d3/roles/36b9d8febf3f4f66b9a27875e9ac1c96` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:52.892600 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4fef0a4c2b740d3beec3b0acb07a7f2, user_id=7170ab9fc27f4ff694bc2ebe056d54d3, role_id=36b9d8febf3f4f66b9a27875e9ac1c96)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:52.895262 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:53.003735 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1446/2894] 10.222.0.41 () {60 vars in 1482 bytes} [Mon Jun 6 20:55:52 2022] PUT /identity/v3/projects/f4fef0a4c2b740d3beec3b0acb07a7f2/users/7170ab9fc27f4ff694bc2ebe056d54d3/roles/36b9d8febf3f4f66b9a27875e9ac1c96 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:53.014042 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] REQUEST_METHOD: `POST` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:53.014333 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:53.014613 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:53.040932 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.auth.core [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] MFA Rules not processed for user `7170ab9fc27f4ff694bc2ebe056d54d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80392) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:53.042161 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:53.103899 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1449/2895] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 06 20:55:53.115409 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:53.117155 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:53.173238 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7b9ad-4f5a-4d72-822c-65d297bec15a admin admin] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:53.176811 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e97bdf-557e-4f2b-98fc-5d18e2ecb9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9110e7fab9084faa94b5d345bb9d519a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:53.177437 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e97bdf-557e-4f2b-98fc-5d18e2ecb9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:53.177724 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e97bdf-557e-4f2b-98fc-5d18e2ecb9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:53.177999 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e97bdf-557e-4f2b-98fc-5d18e2ecb9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:53.186135 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e97bdf-557e-4f2b-98fc-5d18e2ecb9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:53.187540 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e97bdf-557e-4f2b-98fc-5d18e2ecb9e5 service cinder] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:53.196219 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1447/2896] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:55:53.220749 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:53.230938 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:53.282301 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d82a2-a753-4f55-a8dd-abc3c6feddac None None] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:53.287446 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:53.288320 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:53.288603 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:53.288874 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] PATH_INFO: `/v3/users/7170ab9fc27f4ff694bc2ebe056d54d3` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:53.290075 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7170ab9fc27f4ff694bc2ebe056d54d3)` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:53.292014 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:53.473664 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7170ab9fc27f4ff694bc2ebe056d54d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:53.474318 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7170ab9fc27f4ff694bc2ebe056d54d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:53.540942 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7170ab9fc27f4ff694bc2ebe056d54d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:53.547987 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7170ab9fc27f4ff694bc2ebe056d54d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:53.559637 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.notifications [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7170ab9fc27f4ff694bc2ebe056d54d3'} {{(pid=80392) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:53.566216 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1450/2897] 10.222.0.41 () {58 vars in 1223 bytes} [Mon Jun 6 20:55:53 2022] DELETE /identity/v3/users/7170ab9fc27f4ff694bc2ebe056d54d3 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:55:53.570734 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777a01-589a-4062-88a9-aa442763e12e None None] REQUEST_METHOD: `POST` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:53.571024 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777a01-589a-4062-88a9-aa442763e12e None None] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:53.571305 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34777a01-589a-4062-88a9-aa442763e12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:53.588787 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.auth.core [None req-34777a01-589a-4062-88a9-aa442763e12e None None] MFA Rules not processed for user `d4f3ad3806dd4f378cf1552466225045`. Rule list: `[]` (Enabled: `True`). {{(pid=80393) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 06 20:55:53.589972 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-34777a01-589a-4062-88a9-aa442763e12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:53.597843 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:55:53.648936 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1448/2898] 10.222.0.41 () {58 vars in 957 bytes} [Mon Jun 6 20:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 06 20:55:53.658340 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:53.660395 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:53.716585 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c0456-1f8d-4a52-9556-105f727fca68 admin admin] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:53.720212 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e527c9fc7ae1475595c311ac3215b51f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:53.720845 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:53.721128 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:53.721403 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:53.730241 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:53.731598 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:53.740805 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1451/2899] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 20:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 20:55:54.124440 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777a01-589a-4062-88a9-aa442763e12e None None] Authenticating user token {{(pid=80393) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 20:55:54.125856 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.fernet_utils [None req-34777a01-589a-4062-88a9-aa442763e12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80393) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 20:55:54.138984 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34777a01-589a-4062-88a9-aa442763e12e None None] Validating token access rules against request {{(pid=80393) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 20:55:54.143741 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4f3ad3806dd4f378cf1552466225045', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '284b2c60254e43d98184156216a03e15', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80393) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 20:55:54.144614 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 20:55:54.144879 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] SCRIPT_NAME: `/identity` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 20:55:54.145153 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] PATH_INFO: `/v3/projects/f4fef0a4c2b740d3beec3b0acb07a7f2` {{(pid=80393) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 20:55:54.146334 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4fef0a4c2b740d3beec3b0acb07a7f2)` {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 20:55:54.148267 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] RBAC: Authorization granted {{(pid=80393) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 20:55:54.261560 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4fef0a4c2b740d3beec3b0acb07a7f2', 'request_id': 'req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:54.357819 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4fef0a4c2b740d3beec3b0acb07a7f2', 'request_id': 'req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:54.364194 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4fef0a4c2b740d3beec3b0acb07a7f2', 'request_id': 'req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e'} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:54.375169 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] The token cache is being invalidate because project f4fef0a4c2b740d3beec3b0acb07a7f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80393) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 06 20:55:54.375458 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:54.376122 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG keystone.notifications [None req-b03e539b-cc3a-4d6e-97fe-30a7b4cb834e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80393) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 06 20:55:54.377754 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: [pid: 80393|app: 0|req: 1449/2900] 10.222.0.41 () {58 vars in 1232 bytes} [Mon Jun 6 20:55:54 2022] DELETE /identity/v3/projects/f4fef0a4c2b740d3beec3b0acb07a7f2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 06 20:56:03.736682 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 20:56:04.372230 dv-cinder-844879-1-iscsi devstack@keystone.service[80393]: DEBUG dbcounter [-] [80393] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80393) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 06 21:00:27.399200 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] Authenticating user token {{(pid=80392) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 06 21:00:27.400645 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 21:00:27.467962 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb387da-8a56-4d57-9d98-e6fc04136a4e service neutron] Validating token access rules against request {{(pid=80392) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 06 21:00:27.525392 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4dd69635d9a5418e818de21ca309f9a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c821c67b8a14f83b5227073f824d703', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80392) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 06 21:00:27.526035 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] REQUEST_METHOD: `GET` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 06 21:00:27.526321 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] SCRIPT_NAME: `/identity` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 06 21:00:27.526596 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80392) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 06 21:00:27.528373 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.fernet_utils [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80392) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 06 21:00:27.539886 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 06 21:00:27.541450 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a1bd9-575f-42c4-bd13-2bdb7a6a0691 service placement] RBAC: Authorization granted {{(pid=80392) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 06 21:00:27.551725 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: [pid: 80392|app: 0|req: 1452/2901] 10.222.0.41 () {60 vars in 1365 bytes} [Mon Jun 6 21:00:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 06 21:00:37.547179 dv-cinder-844879-1-iscsi devstack@keystone.service[80392]: DEBUG dbcounter [-] [80392] Writing DB stats keystone:SELECT=58 {{(pid=80392) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}